[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 11:36:09 dist-test-slave-7c35 krb5kdc[21570](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 11:36:09 dist-test-slave-7c35 krb5kdc[21570](info): set up 2 sockets
Aug 06 11:36:09 dist-test-slave-7c35 krb5kdc[21570](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260806 11:36:10.225675 21563 mini_kdc.cc:121] Time spent starting KDC: real 1.108s user 0.003s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Aug 06 11:36:10 dist-test-slave-7c35 krb5kdc[21570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016170, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T11:36:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T11:36:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.14.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab.
Entry for principal kudu/127.21.14.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab.
I20260806 11:36:10.383837 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:37333
--webserver_interface=127.21.14.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab
--principal=kudu/127.21.14.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:34655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.14.254:37333
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:36:10.518508 21585 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:36:10.518976 21585 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:36:10.519043 21585 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:36:10.523564 21585 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 11:36:10.523639 21585 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:36:10.523660 21585 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:36:10.523679 21585 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 11:36:10.523694 21585 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 11:36:10.529937 21585 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:34655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.14.254:37333
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab
--principal=kudu/127.21.14.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:37333
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.14.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21585
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:36:10.531867 21585 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:36:10.533229 21585 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 11:36:10.548002 21585 server_base.cc:1034] running on GCE node
W20260806 11:36:10.548034 21590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:10.548034 21591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:10.548275 21593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:10.549372 21585 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:36:10.551515 21585 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:36:10.552760 21585 hybrid_clock.cc:648] HybridClock initialized: now 1786016170552714 us; error 93 us; skew 500 ppm
Aug 06 11:36:10 dist-test-slave-7c35 krb5kdc[21570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016170, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:36:10.556118 21585 init.cc:377] Logged in from keytab as kudu/127.21.14.254@KRBTEST.COM (short username kudu)
I20260806 11:36:10.557538 21585 webserver.cc:460] Webserver started at http://127.21.14.254:42765/ using document root <none> and password file <none>
I20260806 11:36:10.558496 21585 fs_manager.cc:359] Metadata directory not provided
I20260806 11:36:10.558574 21585 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:36:10.559055 21585 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:36:10.562131 21585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/minicluster-data/master-0/data/instance:
uuid: "8cc88a207ab44bd7bbb88623cd2e2a88"
format_stamp: "Formatted at 2026-08-06 11:36:10 on dist-test-slave-7c35"
server_key: "dff62f7c1dfd1a3ebf1295e13132595b"
server_key_iv: "33d064dd84923e4bb6941c59f8328e3a"
server_key_version: "encryptionkey@0"
I20260806 11:36:10.562774 21585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/minicluster-data/master-0/wal/instance:
uuid: "8cc88a207ab44bd7bbb88623cd2e2a88"
format_stamp: "Formatted at 2026-08-06 11:36:10 on dist-test-slave-7c35"
server_key: "dff62f7c1dfd1a3ebf1295e13132595b"
server_key_iv: "33d064dd84923e4bb6941c59f8328e3a"
server_key_version: "encryptionkey@0"
I20260806 11:36:10.568578 21585 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260806 11:36:10.572781 21601 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:10.575218 21585 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.006s sys 0.000s
I20260806 11:36:10.575448 21585 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/minicluster-data/master-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/minicluster-data/master-0/wal
uuid: "8cc88a207ab44bd7bbb88623cd2e2a88"
format_stamp: "Formatted at 2026-08-06 11:36:10 on dist-test-slave-7c35"
server_key: "dff62f7c1dfd1a3ebf1295e13132595b"
server_key_iv: "33d064dd84923e4bb6941c59f8328e3a"
server_key_version: "encryptionkey@0"
I20260806 11:36:10.575621 21585 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:36:10.600390 21585 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:36:10.604856 21585 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:36:10.605085 21585 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:36:10.618599 21585 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.254:37333
I20260806 11:36:10.618739 21655 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.254:37333 every 8 connection(s)
I20260806 11:36:10.620244 21585 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
I20260806 11:36:10.623981 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 21585
I20260806 11:36:10.624224 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/minicluster-data/master-0/wal/instance
I20260806 11:36:10.625895 21563 external_mini_cluster.cc:1428] Setting key f5dc055637d730149538bfcb1b187371
I20260806 11:36:10.626710 21656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:10.639168 21656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cc88a207ab44bd7bbb88623cd2e2a88: Bootstrap starting.
I20260806 11:36:10.644536 21656 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8cc88a207ab44bd7bbb88623cd2e2a88: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:10.645972 21656 log.cc:824] T 00000000000000000000000000000000 P 8cc88a207ab44bd7bbb88623cd2e2a88: Log is configured to *not* fsync() on all Append() calls
I20260806 11:36:10.649701 21656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cc88a207ab44bd7bbb88623cd2e2a88: No bootstrap required, opened a new log
Aug 06 11:36:10 dist-test-slave-7c35 krb5kdc[21570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016170, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:36:10.657224 21656 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8cc88a207ab44bd7bbb88623cd2e2a88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cc88a207ab44bd7bbb88623cd2e2a88" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 37333 } }
I20260806 11:36:10.657593 21656 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8cc88a207ab44bd7bbb88623cd2e2a88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:10.657693 21656 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8cc88a207ab44bd7bbb88623cd2e2a88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cc88a207ab44bd7bbb88623cd2e2a88, State: Initialized, Role: FOLLOWER
I20260806 11:36:10.658691 21656 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8cc88a207ab44bd7bbb88623cd2e2a88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cc88a207ab44bd7bbb88623cd2e2a88" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 37333 } }
I20260806 11:36:10.658988 21656 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8cc88a207ab44bd7bbb88623cd2e2a88 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 11:36:10.659091 21656 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8cc88a207ab44bd7bbb88623cd2e2a88 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 11:36:10.659210 21656 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8cc88a207ab44bd7bbb88623cd2e2a88 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:36:10.660324 21656 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8cc88a207ab44bd7bbb88623cd2e2a88 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cc88a207ab44bd7bbb88623cd2e2a88" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 37333 } }
I20260806 11:36:10.661285 21656 leader_election.cc:302] T 00000000000000000000000000000000 P 8cc88a207ab44bd7bbb88623cd2e2a88 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8cc88a207ab44bd7bbb88623cd2e2a88; no voters:
I20260806 11:36:10.661994 21656 leader_election.cc:288] T 00000000000000000000000000000000 P 8cc88a207ab44bd7bbb88623cd2e2a88 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 11:36:10.662129 21661 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8cc88a207ab44bd7bbb88623cd2e2a88 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:36:10.662422 21661 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8cc88a207ab44bd7bbb88623cd2e2a88 [term 1 LEADER]: Becoming Leader. State: Replica: 8cc88a207ab44bd7bbb88623cd2e2a88, State: Running, Role: LEADER
I20260806 11:36:10.663053 21661 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8cc88a207ab44bd7bbb88623cd2e2a88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cc88a207ab44bd7bbb88623cd2e2a88" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 37333 } }
I20260806 11:36:10.664248 21659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:10.632974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48830 (local address 127.21.14.254:37333)
0806 11:36:10.633485 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:10.633503 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:10.633580 (+ 77us) server_negotiation.cc:407] Connection header received
0806 11:36:10.639366 (+ 5786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:10.639406 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:10.639927 (+ 521us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:10.640303 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:10.645036 (+ 4733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:10.646400 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:10.648479 (+ 2079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:10.649173 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:10.649378 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:10.655503 (+ 6125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:10.655553 (+ 50us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:10.655575 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:10.655623 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:10.658269 (+ 2646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:10.659438 (+ 1169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:10.659445 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:10.659451 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:10.659610 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:10.660071 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:10.660074 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:10.660075 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:10.660707 (+ 632us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:10.661233 (+ 526us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:10.662087 (+ 854us) server_negotiation.cc:299] Negotiation successful
0806 11:36:10.662540 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":167,"threads_started":1}
I20260806 11:36:10.664501 21656 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8cc88a207ab44bd7bbb88623cd2e2a88 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 11:36:10.666824 21662 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cc88a207ab44bd7bbb88623cd2e2a88 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8cc88a207ab44bd7bbb88623cd2e2a88" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cc88a207ab44bd7bbb88623cd2e2a88" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 37333 } } }
I20260806 11:36:10.666810 21663 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cc88a207ab44bd7bbb88623cd2e2a88 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8cc88a207ab44bd7bbb88623cd2e2a88. Latest consensus state: current_term: 1 leader_uuid: "8cc88a207ab44bd7bbb88623cd2e2a88" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cc88a207ab44bd7bbb88623cd2e2a88" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 37333 } } }
I20260806 11:36:10.667090 21662 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cc88a207ab44bd7bbb88623cd2e2a88 [sys.catalog]: This master's current role is: LEADER
I20260806 11:36:10.667127 21663 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cc88a207ab44bd7bbb88623cd2e2a88 [sys.catalog]: This master's current role is: LEADER
I20260806 11:36:10.668617 21675 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 11:36:10.674098 21675 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 11:36:10.684463 21675 catalog_manager.cc:1347] Generated new cluster ID: 1d2bbbbb09f14ae38a0fe25359e3325b
I20260806 11:36:10.684574 21675 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 11:36:10.715530 21675 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 11:36:10.717116 21675 catalog_manager.cc:1504] Loading token signing keys...
I20260806 11:36:10.730441 21675 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8cc88a207ab44bd7bbb88623cd2e2a88: Generated new TSK 0
I20260806 11:36:10.732203 21675 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.14.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab.
Entry for principal kudu/127.21.14.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab.
I20260806 11:36:10.783576 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.193:0
--local_ip_for_outbound_sockets=127.21.14.193
--webserver_interface=127.21.14.193
--webserver_port=0
--tserver_master_addrs=127.21.14.254:37333
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab
--principal=kudu/127.21.14.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:34655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:36:10.910519 21683 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:36:10.910823 21683 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:36:10.910905 21683 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:36:10.915552 21683 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:36:10.915649 21683 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:36:10.915717 21683 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.193
I20260806 11:36:10.921700 21683 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:34655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab
--principal=kudu/127.21.14.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.14.193
--webserver_port=0
--tserver_master_addrs=127.21.14.254:37333
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21683
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.193
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:36:10.922968 21683 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:36:10.924039 21683 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:36:10.935607 21689 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:10.935591 21688 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:10.935786 21683 server_base.cc:1034] running on GCE node
W20260806 11:36:10.935591 21691 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:10.936450 21683 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:36:10.937228 21683 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:36:10.938465 21683 hybrid_clock.cc:648] HybridClock initialized: now 1786016170938448 us; error 35 us; skew 500 ppm
Aug 06 11:36:10 dist-test-slave-7c35 krb5kdc[21570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016170, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:36:10.946529 21683 init.cc:377] Logged in from keytab as kudu/127.21.14.193@KRBTEST.COM (short username kudu)
I20260806 11:36:10.947726 21683 webserver.cc:460] Webserver started at http://127.21.14.193:37013/ using document root <none> and password file <none>
I20260806 11:36:10.948675 21683 fs_manager.cc:359] Metadata directory not provided
I20260806 11:36:10.948771 21683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:36:10.949108 21683 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:36:10.951545 21683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/minicluster-data/ts-0/data/instance:
uuid: "300778d3665a46c9823392d433009e19"
format_stamp: "Formatted at 2026-08-06 11:36:10 on dist-test-slave-7c35"
server_key: "481b70b07e27b6d325c4a541a36b9052"
server_key_iv: "b33518faa2df5ea88dd22bed48244f6d"
server_key_version: "encryptionkey@0"
I20260806 11:36:10.952203 21683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/minicluster-data/ts-0/wal/instance:
uuid: "300778d3665a46c9823392d433009e19"
format_stamp: "Formatted at 2026-08-06 11:36:10 on dist-test-slave-7c35"
server_key: "481b70b07e27b6d325c4a541a36b9052"
server_key_iv: "b33518faa2df5ea88dd22bed48244f6d"
server_key_version: "encryptionkey@0"
I20260806 11:36:10.956780 21683 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 11:36:10.959765 21698 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:10.961236 21683 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 11:36:10.961422 21683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/minicluster-data/ts-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/minicluster-data/ts-0/wal
uuid: "300778d3665a46c9823392d433009e19"
format_stamp: "Formatted at 2026-08-06 11:36:10 on dist-test-slave-7c35"
server_key: "481b70b07e27b6d325c4a541a36b9052"
server_key_iv: "b33518faa2df5ea88dd22bed48244f6d"
server_key_version: "encryptionkey@0"
I20260806 11:36:10.961573 21683 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:36:10.983529 21683 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:36:10.987365 21683 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:36:10.987615 21683 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:36:10.988413 21683 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:36:10.989665 21683 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:36:10.989749 21683 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:10.989831 21683 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:36:10.989885 21683 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:11.001569 21683 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.193:34847
I20260806 11:36:11.001596 21818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.193:34847 every 8 connection(s)
I20260806 11:36:11.002691 21683 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
I20260806 11:36:11.012297 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 21683
I20260806 11:36:11.012497 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/minicluster-data/ts-0/wal/instance
I20260806 11:36:11.012799 21563 external_mini_cluster.cc:1428] Setting key 62315a9a540d9cf90fee8f6b8941ba78
Aug 06 11:36:11 dist-test-slave-7c35 krb5kdc[21570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016170, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.193@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:36:11.020181 21659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:11.005354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:38573 (local address 127.21.14.254:37333)
0806 11:36:11.005552 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:11.005556 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:11.006561 (+ 1005us) server_negotiation.cc:407] Connection header received
0806 11:36:11.008268 (+ 1707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:11.008272 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:11.008341 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:11.008431 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:11.009898 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:11.010491 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:11.011812 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:11.012069 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:11.012194 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:11.015546 (+ 3352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:11.015553 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:11.015559 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:11.015595 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:11.017325 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:11.018106 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:11.018109 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:11.018111 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:11.018218 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:11.018713 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:11.018716 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:11.018718 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:11.019030 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:11.019122 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:11.019884 (+ 762us) server_negotiation.cc:299] Negotiation successful
0806 11:36:11.020031 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 11:36:11.021659 21822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:11.005771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:37333 (local address 127.21.14.193:38573)
0806 11:36:11.006335 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:11.006387 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:11.008002 (+ 1615us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:11.008648 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:11.008656 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:11.009163 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:11.009722 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:11.009738 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:11.010639 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:11.010648 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:11.011668 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:11.011678 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:11.012246 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:11.012250 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:11.012477 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:11.013332 (+ 855us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:11.013352 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:11.015341 (+ 1989us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:11.017470 (+ 2129us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:11.017482 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:11.017508 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:11.017962 (+ 454us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:11.018332 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:11.018339 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:11.018344 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:11.018566 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:11.019150 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:11.019161 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:11.019558 (+ 397us) client_negotiation.cc:769] Sending connection context
0806 11:36:11.019794 (+ 236us) client_negotiation.cc:240] Negotiation successful
0806 11:36:11.020287 (+ 493us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":125,"threads_started":1}
I20260806 11:36:11.023330 21819 heartbeater.cc:347] Connected to a master server at 127.21.14.254:37333
I20260806 11:36:11.023633 21819 heartbeater.cc:464] Registering TS with master...
I20260806 11:36:11.024228 21819 heartbeater.cc:511] Master 127.21.14.254:37333 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.21.14.194@KRBTEST.COM; defaulting to no policy
I20260806 11:36:11.026636 21619 ts_manager.cc:194] Registered new tserver with Master: 300778d3665a46c9823392d433009e19 (127.21.14.193:34847)
I20260806 11:36:11.028292 21619 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.14.193@KRBTEST.COM'} at 127.21.14.193:38573
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab.
Entry for principal kudu/127.21.14.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab.
I20260806 11:36:11.052331 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.194:0
--local_ip_for_outbound_sockets=127.21.14.194
--webserver_interface=127.21.14.194
--webserver_port=0
--tserver_master_addrs=127.21.14.254:37333
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab
--principal=kudu/127.21.14.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:34655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:36:11.185827 21825 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:36:11.186312 21825 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:36:11.186400 21825 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:36:11.191152 21825 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:36:11.191298 21825 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:36:11.191407 21825 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.194
I20260806 11:36:11.197954 21825 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:34655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab
--principal=kudu/127.21.14.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.14.194
--webserver_port=0
--tserver_master_addrs=127.21.14.254:37333
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21825
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.194
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:36:11.199385 21825 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:36:11.200572 21825 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 11:36:11.211692 21825 server_base.cc:1034] running on GCE node
W20260806 11:36:11.211630 21833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:11.211634 21830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:11.211661 21831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:11.212427 21825 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:36:11.213120 21825 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:36:11.214339 21825 hybrid_clock.cc:648] HybridClock initialized: now 1786016171214299 us; error 53 us; skew 500 ppm
Aug 06 11:36:11 dist-test-slave-7c35 krb5kdc[21570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016171, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:36:11.217835 21825 init.cc:377] Logged in from keytab as kudu/127.21.14.194@KRBTEST.COM (short username kudu)
I20260806 11:36:11.218998 21825 webserver.cc:460] Webserver started at http://127.21.14.194:35993/ using document root <none> and password file <none>
I20260806 11:36:11.219611 21825 fs_manager.cc:359] Metadata directory not provided
I20260806 11:36:11.219677 21825 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:36:11.219861 21825 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:36:11.222198 21825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/minicluster-data/ts-1/data/instance:
uuid: "8b06f833ce374147a158a6ab504b243f"
format_stamp: "Formatted at 2026-08-06 11:36:11 on dist-test-slave-7c35"
server_key: "860688531ea15c124b0a22c3213525b2"
server_key_iv: "65b9ed508a26dbfa40b1b765650532a8"
server_key_version: "encryptionkey@0"
I20260806 11:36:11.222826 21825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/minicluster-data/ts-1/wal/instance:
uuid: "8b06f833ce374147a158a6ab504b243f"
format_stamp: "Formatted at 2026-08-06 11:36:11 on dist-test-slave-7c35"
server_key: "860688531ea15c124b0a22c3213525b2"
server_key_iv: "65b9ed508a26dbfa40b1b765650532a8"
server_key_version: "encryptionkey@0"
I20260806 11:36:11.227337 21825 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 11:36:11.230149 21840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:11.231791 21825 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 11:36:11.231920 21825 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/minicluster-data/ts-1/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/minicluster-data/ts-1/wal
uuid: "8b06f833ce374147a158a6ab504b243f"
format_stamp: "Formatted at 2026-08-06 11:36:11 on dist-test-slave-7c35"
server_key: "860688531ea15c124b0a22c3213525b2"
server_key_iv: "65b9ed508a26dbfa40b1b765650532a8"
server_key_version: "encryptionkey@0"
I20260806 11:36:11.232034 21825 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:36:11.244815 21825 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:36:11.248689 21825 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:36:11.248919 21825 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:36:11.249788 21825 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:36:11.250824 21825 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:36:11.250969 21825 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:11.251047 21825 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:36:11.251099 21825 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:11.262785 21825 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.194:45679
I20260806 11:36:11.263028 21961 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.194:45679 every 8 connection(s)
I20260806 11:36:11.263878 21825 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
I20260806 11:36:11.271562 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 21825
I20260806 11:36:11.271658 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/minicluster-data/ts-1/wal/instance
I20260806 11:36:11.271917 21563 external_mini_cluster.cc:1428] Setting key ac2ca279348b7638612008e90b1f0f98
Aug 06 11:36:11 dist-test-slave-7c35 krb5kdc[21570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016171, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.194@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:36:11.280719 21659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:11.266049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:56913 (local address 127.21.14.254:37333)
0806 11:36:11.266198 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:11.266202 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:11.267376 (+ 1174us) server_negotiation.cc:407] Connection header received
0806 11:36:11.268833 (+ 1457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:11.268837 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:11.268902 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:11.268992 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:11.270356 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:11.271033 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:11.272815 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:11.273142 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:11.273266 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:11.276527 (+ 3261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:11.276531 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:11.276533 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:11.276565 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:11.278182 (+ 1617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:11.278810 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:11.278813 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:11.278815 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:11.278962 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:11.279366 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:11.279369 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:11.279371 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:11.279616 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:11.279731 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:11.280378 (+ 647us) server_negotiation.cc:299] Negotiation successful
0806 11:36:11.280542 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 11:36:11.281734 21965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:11.266592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:37333 (local address 127.21.14.194:56913)
0806 11:36:11.267206 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:11.267255 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:11.268580 (+ 1325us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:11.269191 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:11.269199 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:11.269637 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:11.270189 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:11.270204 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:11.271173 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:11.271178 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:11.272627 (+ 1449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:11.272642 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:11.273360 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:11.273369 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:11.273676 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:11.274413 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:11.274431 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:11.276287 (+ 1856us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:11.278349 (+ 2062us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:11.278357 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:11.278377 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:11.278687 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:11.279073 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:11.279076 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:11.279078 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:11.279250 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:11.279733 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:11.279742 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:11.280046 (+ 304us) client_negotiation.cc:769] Sending connection context
0806 11:36:11.280333 (+ 287us) client_negotiation.cc:240] Negotiation successful
0806 11:36:11.280738 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":215,"threads_started":1}
WARNING: no policy specified for kudu/127.21.14.195@KRBTEST.COM; defaulting to no policy
I20260806 11:36:11.283217 21962 heartbeater.cc:347] Connected to a master server at 127.21.14.254:37333
I20260806 11:36:11.283516 21962 heartbeater.cc:464] Registering TS with master...
I20260806 11:36:11.284130 21962 heartbeater.cc:511] Master 127.21.14.254:37333 requested a full tablet report, sending...
I20260806 11:36:11.285477 21619 ts_manager.cc:194] Registered new tserver with Master: 8b06f833ce374147a158a6ab504b243f (127.21.14.194:45679)
I20260806 11:36:11.286147 21619 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.14.194@KRBTEST.COM'} at 127.21.14.194:56913
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab.
Entry for principal kudu/127.21.14.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab.
I20260806 11:36:11.307823 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.195:0
--local_ip_for_outbound_sockets=127.21.14.195
--webserver_interface=127.21.14.195
--webserver_port=0
--tserver_master_addrs=127.21.14.254:37333
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab
--principal=kudu/127.21.14.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:34655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:36:11.438138 21969 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:36:11.438529 21969 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:36:11.438616 21969 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:36:11.443277 21969 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:36:11.443408 21969 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:36:11.443522 21969 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.195
I20260806 11:36:11.450109 21969 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:34655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab
--principal=kudu/127.21.14.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.14.195
--webserver_port=0
--tserver_master_addrs=127.21.14.254:37333
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21969
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.195
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:36:11.451502 21969 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:36:11.452678 21969 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:36:11.464162 21975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:11.464234 21977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:11.464272 21974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:11.464547 21969 server_base.cc:1034] running on GCE node
I20260806 11:36:11.465130 21969 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:36:11.466101 21969 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:36:11.467285 21969 hybrid_clock.cc:648] HybridClock initialized: now 1786016171467258 us; error 42 us; skew 500 ppm
Aug 06 11:36:11 dist-test-slave-7c35 krb5kdc[21570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016171, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:36:11.470400 21969 init.cc:377] Logged in from keytab as kudu/127.21.14.195@KRBTEST.COM (short username kudu)
I20260806 11:36:11.471575 21969 webserver.cc:460] Webserver started at http://127.21.14.195:34651/ using document root <none> and password file <none>
I20260806 11:36:11.472193 21969 fs_manager.cc:359] Metadata directory not provided
I20260806 11:36:11.472258 21969 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:36:11.472442 21969 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:36:11.474670 21969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/minicluster-data/ts-2/data/instance:
uuid: "7acb293593444533b6c1ed8155002c5c"
format_stamp: "Formatted at 2026-08-06 11:36:11 on dist-test-slave-7c35"
server_key: "8600144e401ea0263f9ed2c821dc3ed8"
server_key_iv: "e400746440776846a616fe4d8580afa6"
server_key_version: "encryptionkey@0"
I20260806 11:36:11.475325 21969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/minicluster-data/ts-2/wal/instance:
uuid: "7acb293593444533b6c1ed8155002c5c"
format_stamp: "Formatted at 2026-08-06 11:36:11 on dist-test-slave-7c35"
server_key: "8600144e401ea0263f9ed2c821dc3ed8"
server_key_iv: "e400746440776846a616fe4d8580afa6"
server_key_version: "encryptionkey@0"
I20260806 11:36:11.479599 21969 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 11:36:11.482769 21984 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:11.484339 21969 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 11:36:11.484549 21969 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/minicluster-data/ts-2/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/minicluster-data/ts-2/wal
uuid: "7acb293593444533b6c1ed8155002c5c"
format_stamp: "Formatted at 2026-08-06 11:36:11 on dist-test-slave-7c35"
server_key: "8600144e401ea0263f9ed2c821dc3ed8"
server_key_iv: "e400746440776846a616fe4d8580afa6"
server_key_version: "encryptionkey@0"
I20260806 11:36:11.484690 21969 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:36:11.497258 21969 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:36:11.501551 21969 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:36:11.501811 21969 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:36:11.502584 21969 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:36:11.503597 21969 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:36:11.503648 21969 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:11.503716 21969 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:36:11.503749 21969 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:11.516191 21969 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.195:39547
I20260806 11:36:11.516320 22100 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.195:39547 every 8 connection(s)
I20260806 11:36:11.517459 21969 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
I20260806 11:36:11.527251 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 21969
I20260806 11:36:11.527413 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786016169045816-21563-0/minicluster-data/ts-2/wal/instance
I20260806 11:36:11.527720 21563 external_mini_cluster.cc:1428] Setting key ac2a3e646a348a0c15b4f8e20bf614f2
Aug 06 11:36:11 dist-test-slave-7c35 krb5kdc[21570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016171, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.195@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:36:11.534617 21659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:11.519665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:50279 (local address 127.21.14.254:37333)
0806 11:36:11.519831 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:11.519835 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:11.521460 (+ 1625us) server_negotiation.cc:407] Connection header received
0806 11:36:11.522518 (+ 1058us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:11.522522 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:11.522591 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:11.522709 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:11.524206 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:11.524783 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:11.526134 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:11.526383 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:11.526514 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:11.530113 (+ 3599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:11.530116 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:11.530118 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:11.530150 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:11.532027 (+ 1877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:11.532651 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:11.532654 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:11.532656 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:11.532752 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:11.533310 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:11.533314 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:11.533316 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:11.533545 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:11.533663 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:11.534281 (+ 618us) server_negotiation.cc:299] Negotiation successful
0806 11:36:11.534433 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 11:36:11.535647 22104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:11.520131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:37333 (local address 127.21.14.195:50279)
0806 11:36:11.521259 (+ 1128us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:11.521314 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:11.522255 (+ 941us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:11.522978 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:11.522987 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:11.523414 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:11.524004 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:11.524022 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:11.524947 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:11.524958 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:11.525967 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:11.525980 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:11.526588 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:11.526593 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:11.526793 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:11.527662 (+ 869us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:11.527682 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:11.529885 (+ 2203us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:11.532182 (+ 2297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:11.532190 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:11.532214 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:11.532532 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:11.532874 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:11.532877 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:11.532879 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:11.533201 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:11.533666 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:11.533677 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:11.534008 (+ 331us) client_negotiation.cc:769] Sending connection context
0806 11:36:11.534235 (+ 227us) client_negotiation.cc:240] Negotiation successful
0806 11:36:11.534624 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":886,"thread_start_us":693,"threads_started":1}
I20260806 11:36:11.537267 22101 heartbeater.cc:347] Connected to a master server at 127.21.14.254:37333
I20260806 11:36:11.537576 22101 heartbeater.cc:464] Registering TS with master...
I20260806 11:36:11.538152 22101 heartbeater.cc:511] Master 127.21.14.254:37333 requested a full tablet report, sending...
I20260806 11:36:11.539561 21619 ts_manager.cc:194] Registered new tserver with Master: 7acb293593444533b6c1ed8155002c5c (127.21.14.195:39547)
I20260806 11:36:11.540393 21619 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.14.195@KRBTEST.COM'} at 127.21.14.195:50279
I20260806 11:36:11.543718 21563 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 11:36:11 dist-test-slave-7c35 krb5kdc[21570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016171, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 11:36:11 dist-test-slave-7c35 krb5kdc[21570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016171, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.14.193@KRBTEST.COM
I20260806 11:36:11.576237 22110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:11.563946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46488 (local address 127.21.14.193:34847)
0806 11:36:11.564312 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:11.564319 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:11.564347 (+ 28us) server_negotiation.cc:407] Connection header received
0806 11:36:11.564585 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:11.564591 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:11.564801 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:11.564924 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:11.565838 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:11.566750 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:11.568068 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:11.568522 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:11.568625 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:11.571606 (+ 2981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:11.571620 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:11.571637 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:11.571670 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:11.573530 (+ 1860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:11.574110 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:11.574117 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:11.574119 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:11.574223 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:11.574740 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:11.574744 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:11.574745 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:11.575264 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:11.575453 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:11.575796 (+ 343us) server_negotiation.cc:299] Negotiation successful
0806 11:36:11.576060 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":130,"threads_started":1}
W20260806 11:36:11.577545 21727 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46488
Aug 06 11:36:11 dist-test-slave-7c35 krb5kdc[21570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016171, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 11:36:11 dist-test-slave-7c35 krb5kdc[21570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016171, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.14.193@KRBTEST.COM
I20260806 11:36:11.608587 22110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:11.598420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46498 (local address 127.21.14.193:34847)
0806 11:36:11.598622 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:11.598628 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:11.598775 (+ 147us) server_negotiation.cc:407] Connection header received
0806 11:36:11.598956 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:11.598959 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:11.599008 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:11.599085 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:11.599909 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:11.600467 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:11.601360 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:11.601611 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:11.601715 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:11.604763 (+ 3048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:11.604767 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:11.604769 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:11.604802 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:11.606426 (+ 1624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:11.607029 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:11.607033 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:11.607034 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:11.607131 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:11.607517 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:11.607521 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:11.607522 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:11.607825 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:11.607946 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:11.608314 (+ 368us) server_negotiation.cc:299] Negotiation successful
0806 11:36:11.608451 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260806 11:36:11.610056 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 21683
I20260806 11:36:11.621088 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 21825
I20260806 11:36:11.629148 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 21969
I20260806 11:36:11.638062 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 21585
2026-08-06T11:36:11Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2562 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 11:36:11 dist-test-slave-7c35 krb5kdc[22117](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 11:36:11 dist-test-slave-7c35 krb5kdc[22117](info): set up 2 sockets
Aug 06 11:36:11 dist-test-slave-7c35 krb5kdc[22117](info): commencing operation
krb5kdc: starting...
W20260806 11:36:12.894025 21563 mini_kdc.cc:121] Time spent starting KDC: real 1.230s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Aug 06 11:36:12 dist-test-slave-7c35 krb5kdc[22117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016172, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T11:36:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T11:36:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.14.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab.
Entry for principal kudu/127.21.14.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab.
I20260806 11:36:13.036007 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:40221
--webserver_interface=127.21.14.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab
--principal=kudu/127.21.14.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:33149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.14.254:40221
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:36:13.162070 22130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:36:13.162464 22130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:36:13.162556 22130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:36:13.167652 22130 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 11:36:13.167804 22130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:36:13.167841 22130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:36:13.167923 22130 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 11:36:13.167974 22130 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 11:36:13.174346 22130 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:33149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.14.254:40221
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab
--principal=kudu/127.21.14.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:40221
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.14.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22130
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:36:13.176028 22130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:36:13.177302 22130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:36:13.188198 22139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:13.188198 22136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:13.188225 22137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:13.189535 22130 server_base.cc:1034] running on GCE node
I20260806 11:36:13.190340 22130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:36:13.191777 22130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:36:13.193015 22130 hybrid_clock.cc:648] HybridClock initialized: now 1786016173192978 us; error 69 us; skew 500 ppm
Aug 06 11:36:13 dist-test-slave-7c35 krb5kdc[22117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016173, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:36:13.196692 22130 init.cc:377] Logged in from keytab as kudu/127.21.14.254@KRBTEST.COM (short username kudu)
I20260806 11:36:13.197925 22130 webserver.cc:460] Webserver started at http://127.21.14.254:46647/ using document root <none> and password file <none>
I20260806 11:36:13.198613 22130 fs_manager.cc:359] Metadata directory not provided
I20260806 11:36:13.198685 22130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:36:13.198985 22130 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:36:13.201685 22130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/minicluster-data/master-0/data/instance:
uuid: "fef32e01a083468791b41dc97bf85648"
format_stamp: "Formatted at 2026-08-06 11:36:13 on dist-test-slave-7c35"
server_key: "182e653e3081c50439f00880a47a8887"
server_key_iv: "3472edd902ee3d03f42ba202d163ce1d"
server_key_version: "encryptionkey@0"
I20260806 11:36:13.202457 22130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/minicluster-data/master-0/wal/instance:
uuid: "fef32e01a083468791b41dc97bf85648"
format_stamp: "Formatted at 2026-08-06 11:36:13 on dist-test-slave-7c35"
server_key: "182e653e3081c50439f00880a47a8887"
server_key_iv: "3472edd902ee3d03f42ba202d163ce1d"
server_key_version: "encryptionkey@0"
I20260806 11:36:13.207265 22130 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260806 11:36:13.210444 22146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:13.212291 22130 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 11:36:13.212532 22130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/minicluster-data/master-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/minicluster-data/master-0/wal
uuid: "fef32e01a083468791b41dc97bf85648"
format_stamp: "Formatted at 2026-08-06 11:36:13 on dist-test-slave-7c35"
server_key: "182e653e3081c50439f00880a47a8887"
server_key_iv: "3472edd902ee3d03f42ba202d163ce1d"
server_key_version: "encryptionkey@0"
I20260806 11:36:13.212698 22130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:36:13.259207 22130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:36:13.264529 22130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:36:13.264837 22130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:36:13.274245 22130 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.254:40221
I20260806 11:36:13.274330 22199 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.254:40221 every 8 connection(s)
I20260806 11:36:13.275683 22130 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
I20260806 11:36:13.279141 22200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:13.285993 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 22130
I20260806 11:36:13.285995 22200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fef32e01a083468791b41dc97bf85648: Bootstrap starting.
I20260806 11:36:13.286134 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/minicluster-data/master-0/wal/instance
I20260806 11:36:13.286414 21563 external_mini_cluster.cc:1428] Setting key 32044f141aabef2e13da22aa8e50a2ad
I20260806 11:36:13.289559 22200 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fef32e01a083468791b41dc97bf85648: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:13.290555 22200 log.cc:824] T 00000000000000000000000000000000 P fef32e01a083468791b41dc97bf85648: Log is configured to *not* fsync() on all Append() calls
I20260806 11:36:13.293164 22200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fef32e01a083468791b41dc97bf85648: No bootstrap required, opened a new log
Aug 06 11:36:13 dist-test-slave-7c35 krb5kdc[22117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016172, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:36:13.298731 22200 raft_consensus.cc:348] T 00000000000000000000000000000000 P fef32e01a083468791b41dc97bf85648 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fef32e01a083468791b41dc97bf85648" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 40221 } }
I20260806 11:36:13.299166 22200 raft_consensus.cc:374] T 00000000000000000000000000000000 P fef32e01a083468791b41dc97bf85648 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:13.299247 22200 raft_consensus.cc:729] T 00000000000000000000000000000000 P fef32e01a083468791b41dc97bf85648 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fef32e01a083468791b41dc97bf85648, State: Initialized, Role: FOLLOWER
I20260806 11:36:13.299860 22200 consensus_queue.cc:260] T 00000000000000000000000000000000 P fef32e01a083468791b41dc97bf85648 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fef32e01a083468791b41dc97bf85648" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 40221 } }
I20260806 11:36:13.300045 22200 raft_consensus.cc:388] T 00000000000000000000000000000000 P fef32e01a083468791b41dc97bf85648 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 11:36:13.300120 22200 raft_consensus.cc:482] T 00000000000000000000000000000000 P fef32e01a083468791b41dc97bf85648 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 11:36:13.300246 22200 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fef32e01a083468791b41dc97bf85648 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:36:13.301640 22200 raft_consensus.cc:504] T 00000000000000000000000000000000 P fef32e01a083468791b41dc97bf85648 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fef32e01a083468791b41dc97bf85648" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 40221 } }
I20260806 11:36:13.302085 22200 leader_election.cc:302] T 00000000000000000000000000000000 P fef32e01a083468791b41dc97bf85648 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fef32e01a083468791b41dc97bf85648; no voters:
I20260806 11:36:13.302485 22200 leader_election.cc:288] T 00000000000000000000000000000000 P fef32e01a083468791b41dc97bf85648 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 11:36:13.302577 22205 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fef32e01a083468791b41dc97bf85648 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:36:13.302807 22205 raft_consensus.cc:686] T 00000000000000000000000000000000 P fef32e01a083468791b41dc97bf85648 [term 1 LEADER]: Becoming Leader. State: Replica: fef32e01a083468791b41dc97bf85648, State: Running, Role: LEADER
I20260806 11:36:13.303184 22205 consensus_queue.cc:237] T 00000000000000000000000000000000 P fef32e01a083468791b41dc97bf85648 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fef32e01a083468791b41dc97bf85648" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 40221 } }
I20260806 11:36:13.304075 22200 sys_catalog.cc:564] T 00000000000000000000000000000000 P fef32e01a083468791b41dc97bf85648 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 11:36:13.304582 22203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:13.288270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43424 (local address 127.21.14.254:40221)
0806 11:36:13.288766 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:13.288785 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:13.288829 (+ 44us) server_negotiation.cc:407] Connection header received
0806 11:36:13.289582 (+ 753us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:13.289602 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:13.290051 (+ 449us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:13.290386 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:13.291445 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:13.292564 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:13.293764 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:13.294302 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:13.294458 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:13.297573 (+ 3115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:13.297581 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:13.297606 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:13.297659 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:13.299942 (+ 2283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:13.300682 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:13.300689 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:13.300696 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:13.300858 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:13.301169 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:13.301173 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:13.301174 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:13.301837 (+ 663us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:13.302055 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:13.302496 (+ 441us) server_negotiation.cc:299] Negotiation successful
0806 11:36:13.303319 (+ 823us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":162,"threads_started":1}
I20260806 11:36:13.305060 22207 sys_catalog.cc:455] T 00000000000000000000000000000000 P fef32e01a083468791b41dc97bf85648 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fef32e01a083468791b41dc97bf85648" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fef32e01a083468791b41dc97bf85648" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 40221 } } }
I20260806 11:36:13.305212 22208 sys_catalog.cc:455] T 00000000000000000000000000000000 P fef32e01a083468791b41dc97bf85648 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fef32e01a083468791b41dc97bf85648. Latest consensus state: current_term: 1 leader_uuid: "fef32e01a083468791b41dc97bf85648" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fef32e01a083468791b41dc97bf85648" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 40221 } } }
I20260806 11:36:13.305323 22208 sys_catalog.cc:458] T 00000000000000000000000000000000 P fef32e01a083468791b41dc97bf85648 [sys.catalog]: This master's current role is: LEADER
I20260806 11:36:13.305327 22207 sys_catalog.cc:458] T 00000000000000000000000000000000 P fef32e01a083468791b41dc97bf85648 [sys.catalog]: This master's current role is: LEADER
I20260806 11:36:13.305819 22215 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 11:36:13.309947 22215 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 11:36:13.317411 22215 catalog_manager.cc:1347] Generated new cluster ID: f8899b13cb9943afbe83c7926f8b00b9
I20260806 11:36:13.317523 22215 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 11:36:13.355204 22215 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 11:36:13.356323 22215 catalog_manager.cc:1504] Loading token signing keys...
I20260806 11:36:13.363730 22215 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fef32e01a083468791b41dc97bf85648: Generated new TSK 0
I20260806 11:36:13.364626 22215 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.14.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab.
Entry for principal kudu/127.21.14.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab.
I20260806 11:36:13.406584 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.193:0
--local_ip_for_outbound_sockets=127.21.14.193
--webserver_interface=127.21.14.193
--webserver_port=0
--tserver_master_addrs=127.21.14.254:40221
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab
--principal=kudu/127.21.14.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:33149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:36:13.540591 22228 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:36:13.540930 22228 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:36:13.540978 22228 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:36:13.545305 22228 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:36:13.545385 22228 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:36:13.545451 22228 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.193
I20260806 11:36:13.551774 22228 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:33149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab
--principal=kudu/127.21.14.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.14.193
--webserver_port=0
--tserver_master_addrs=127.21.14.254:40221
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22228
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.193
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:36:13.552980 22228 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:36:13.554066 22228 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:36:13.564743 22237 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:13.564735 22234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:13.564743 22235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:13.565013 22228 server_base.cc:1034] running on GCE node
I20260806 11:36:13.565621 22228 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:36:13.566383 22228 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:36:13.567565 22228 hybrid_clock.cc:648] HybridClock initialized: now 1786016173567526 us; error 64 us; skew 500 ppm
Aug 06 11:36:13 dist-test-slave-7c35 krb5kdc[22117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016173, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:36:13.570720 22228 init.cc:377] Logged in from keytab as kudu/127.21.14.193@KRBTEST.COM (short username kudu)
I20260806 11:36:13.571831 22228 webserver.cc:460] Webserver started at http://127.21.14.193:37469/ using document root <none> and password file <none>
I20260806 11:36:13.572460 22228 fs_manager.cc:359] Metadata directory not provided
I20260806 11:36:13.572528 22228 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:36:13.572785 22228 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:36:13.575089 22228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/minicluster-data/ts-0/data/instance:
uuid: "08ad0378c4ed4dbbac3df4e736e9354a"
format_stamp: "Formatted at 2026-08-06 11:36:13 on dist-test-slave-7c35"
server_key: "57ba4a6e95e6f18dd7db4345ad301a67"
server_key_iv: "ef6118286b7050f8529cfd157cd14ac6"
server_key_version: "encryptionkey@0"
I20260806 11:36:13.575726 22228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/minicluster-data/ts-0/wal/instance:
uuid: "08ad0378c4ed4dbbac3df4e736e9354a"
format_stamp: "Formatted at 2026-08-06 11:36:13 on dist-test-slave-7c35"
server_key: "57ba4a6e95e6f18dd7db4345ad301a67"
server_key_iv: "ef6118286b7050f8529cfd157cd14ac6"
server_key_version: "encryptionkey@0"
I20260806 11:36:13.580183 22228 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 11:36:13.583488 22244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:13.585150 22228 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 11:36:13.585399 22228 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/minicluster-data/ts-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/minicluster-data/ts-0/wal
uuid: "08ad0378c4ed4dbbac3df4e736e9354a"
format_stamp: "Formatted at 2026-08-06 11:36:13 on dist-test-slave-7c35"
server_key: "57ba4a6e95e6f18dd7db4345ad301a67"
server_key_iv: "ef6118286b7050f8529cfd157cd14ac6"
server_key_version: "encryptionkey@0"
I20260806 11:36:13.585564 22228 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:36:13.605940 22228 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:36:13.610047 22228 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:36:13.610330 22228 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:36:13.611057 22228 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:36:13.612063 22228 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:36:13.612115 22228 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:13.612159 22228 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:36:13.612174 22228 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:13.624503 22228 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.193:39883
I20260806 11:36:13.624818 22361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.193:39883 every 8 connection(s)
I20260806 11:36:13.625710 22228 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
I20260806 11:36:13.636149 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 22228
I20260806 11:36:13.636257 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/minicluster-data/ts-0/wal/instance
I20260806 11:36:13.636538 21563 external_mini_cluster.cc:1428] Setting key 7d906044bfccdba7fdf1696f871a304d
Aug 06 11:36:13 dist-test-slave-7c35 krb5kdc[22117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016173, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.193@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:36:13.642623 22203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:13.628095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:41229 (local address 127.21.14.254:40221)
0806 11:36:13.628298 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:13.628302 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:13.629681 (+ 1379us) server_negotiation.cc:407] Connection header received
0806 11:36:13.630931 (+ 1250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:13.630935 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:13.631012 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:13.631111 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:13.632539 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:13.633108 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:13.634434 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:13.634687 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:13.634807 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:13.638039 (+ 3232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:13.638046 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:13.638052 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:13.638107 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:13.640028 (+ 1921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:13.640789 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:13.640792 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:13.640793 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:13.640880 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:13.641337 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:13.641340 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:13.641341 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:13.641533 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:13.641610 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:13.642363 (+ 753us) server_negotiation.cc:299] Negotiation successful
0806 11:36:13.642483 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260806 11:36:13.644100 22364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:13.628616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:40221 (local address 127.21.14.193:41229)
0806 11:36:13.629476 (+ 860us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:13.629529 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:13.630601 (+ 1072us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:13.631329 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:13.631338 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:13.631774 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:13.632331 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:13.632349 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:13.633264 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:13.633268 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:13.634264 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:13.634275 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:13.634907 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:13.634912 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:13.635111 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:13.635853 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:13.635873 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:13.637785 (+ 1912us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:13.640208 (+ 2423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:13.640215 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:13.640233 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:13.640634 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:13.640983 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:13.640987 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:13.640990 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:13.641230 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:13.641642 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:13.641650 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:13.642080 (+ 430us) client_negotiation.cc:769] Sending connection context
0806 11:36:13.642332 (+ 252us) client_negotiation.cc:240] Negotiation successful
0806 11:36:13.642755 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":612,"thread_start_us":150,"threads_started":1}
I20260806 11:36:13.645622 22362 heartbeater.cc:347] Connected to a master server at 127.21.14.254:40221
I20260806 11:36:13.645895 22362 heartbeater.cc:464] Registering TS with master...
I20260806 11:36:13.646462 22362 heartbeater.cc:511] Master 127.21.14.254:40221 requested a full tablet report, sending...
I20260806 11:36:13.648216 22163 ts_manager.cc:194] Registered new tserver with Master: 08ad0378c4ed4dbbac3df4e736e9354a (127.21.14.193:39883)
WARNING: no policy specified for kudu/127.21.14.194@KRBTEST.COM; defaulting to no policy
I20260806 11:36:13.649426 22163 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.14.193@KRBTEST.COM'} at 127.21.14.193:41229
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab.
Entry for principal kudu/127.21.14.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab.
I20260806 11:36:13.674649 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.194:0
--local_ip_for_outbound_sockets=127.21.14.194
--webserver_interface=127.21.14.194
--webserver_port=0
--tserver_master_addrs=127.21.14.254:40221
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab
--principal=kudu/127.21.14.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:33149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:36:13.796299 22367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:36:13.796676 22367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:36:13.796766 22367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:36:13.801411 22367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:36:13.801499 22367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:36:13.801594 22367 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.194
I20260806 11:36:13.807969 22367 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:33149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab
--principal=kudu/127.21.14.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.14.194
--webserver_port=0
--tserver_master_addrs=127.21.14.254:40221
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22367
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.194
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:36:13.809270 22367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:36:13.810431 22367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:36:13.821780 22372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:13.821808 22376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:13.821784 22374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:13.822273 22367 server_base.cc:1034] running on GCE node
I20260806 11:36:13.822751 22367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:36:13.823576 22367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:36:13.824795 22367 hybrid_clock.cc:648] HybridClock initialized: now 1786016173824772 us; error 33 us; skew 500 ppm
Aug 06 11:36:13 dist-test-slave-7c35 krb5kdc[22117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016173, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:36:13.827943 22367 init.cc:377] Logged in from keytab as kudu/127.21.14.194@KRBTEST.COM (short username kudu)
I20260806 11:36:13.829032 22367 webserver.cc:460] Webserver started at http://127.21.14.194:39837/ using document root <none> and password file <none>
I20260806 11:36:13.829676 22367 fs_manager.cc:359] Metadata directory not provided
I20260806 11:36:13.829773 22367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:36:13.830039 22367 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:36:13.832414 22367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/minicluster-data/ts-1/data/instance:
uuid: "db49f67cc2c14f3ba007fabc0acb7b8d"
format_stamp: "Formatted at 2026-08-06 11:36:13 on dist-test-slave-7c35"
server_key: "2ffd97c6f2aa390accf56cd0701db9e0"
server_key_iv: "a05b21aacd424d78088db160cdaa7ad0"
server_key_version: "encryptionkey@0"
I20260806 11:36:13.833057 22367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/minicluster-data/ts-1/wal/instance:
uuid: "db49f67cc2c14f3ba007fabc0acb7b8d"
format_stamp: "Formatted at 2026-08-06 11:36:13 on dist-test-slave-7c35"
server_key: "2ffd97c6f2aa390accf56cd0701db9e0"
server_key_iv: "a05b21aacd424d78088db160cdaa7ad0"
server_key_version: "encryptionkey@0"
I20260806 11:36:13.837419 22367 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 11:36:13.840449 22383 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:13.841845 22367 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 11:36:13.842024 22367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/minicluster-data/ts-1/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/minicluster-data/ts-1/wal
uuid: "db49f67cc2c14f3ba007fabc0acb7b8d"
format_stamp: "Formatted at 2026-08-06 11:36:13 on dist-test-slave-7c35"
server_key: "2ffd97c6f2aa390accf56cd0701db9e0"
server_key_iv: "a05b21aacd424d78088db160cdaa7ad0"
server_key_version: "encryptionkey@0"
I20260806 11:36:13.842161 22367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:36:13.864670 22367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:36:13.868505 22367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:36:13.868736 22367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:36:13.869676 22367 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:36:13.870690 22367 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:36:13.870765 22367 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:13.870838 22367 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:36:13.870901 22367 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260806 11:36:13.880455 22367 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.194:46161
I20260806 11:36:13.880498 22501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.194:46161 every 8 connection(s)
I20260806 11:36:13.881476 22367 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
I20260806 11:36:13.882704 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 22367
I20260806 11:36:13.882787 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/minicluster-data/ts-1/wal/instance
I20260806 11:36:13.883085 21563 external_mini_cluster.cc:1428] Setting key 05d7bdecd8801320e6df46fa5a3793ca
WARNING: no policy specified for kudu/127.21.14.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.195@KRBTEST.COM" created.
Aug 06 11:36:13 dist-test-slave-7c35 krb5kdc[22117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016173, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.194@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:36:13.902813 22203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:13.883742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:45271 (local address 127.21.14.254:40221)
0806 11:36:13.883910 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:13.883915 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:13.885471 (+ 1556us) server_negotiation.cc:407] Connection header received
0806 11:36:13.887243 (+ 1772us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:13.887249 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:13.887331 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:13.887486 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:13.888988 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:13.889722 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:13.891543 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:13.891834 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:13.891981 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:13.898432 (+ 6451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:13.898437 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:13.898439 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:13.898496 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:13.900224 (+ 1728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:13.901002 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:13.901009 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:13.901013 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:13.901122 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:13.901505 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:13.901511 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:13.901516 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:13.901812 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:13.901943 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:13.902487 (+ 544us) server_negotiation.cc:299] Negotiation successful
0806 11:36:13.902635 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 11:36:13.903855 22505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:13.884246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:40221 (local address 127.21.14.194:45271)
0806 11:36:13.885206 (+ 960us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:13.885282 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:13.886831 (+ 1549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:13.887708 (+ 877us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:13.887717 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:13.888170 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:13.888764 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:13.888782 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:13.889902 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:13.889907 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:13.891363 (+ 1456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:13.891378 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:13.892045 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:13.892050 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:13.892296 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:13.893083 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:13.893105 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:13.898124 (+ 5019us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:13.900395 (+ 2271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:13.900404 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:13.900441 (+ 37us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:13.900839 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:13.901223 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:13.901225 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:13.901227 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:13.901383 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:13.901926 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:13.901933 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:13.902210 (+ 277us) client_negotiation.cc:769] Sending connection context
0806 11:36:13.902412 (+ 202us) client_negotiation.cc:240] Negotiation successful
0806 11:36:13.902834 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":691,"thread_start_us":159,"threads_started":1}
I20260806 11:36:13.905257 22502 heartbeater.cc:347] Connected to a master server at 127.21.14.254:40221
I20260806 11:36:13.905537 22502 heartbeater.cc:464] Registering TS with master...
I20260806 11:36:13.906183 22502 heartbeater.cc:511] Master 127.21.14.254:40221 requested a full tablet report, sending...
I20260806 11:36:13.907492 22163 ts_manager.cc:194] Registered new tserver with Master: db49f67cc2c14f3ba007fabc0acb7b8d (127.21.14.194:46161)
I20260806 11:36:13.908182 22163 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.14.194@KRBTEST.COM'} at 127.21.14.194:45271
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab.
Entry for principal kudu/127.21.14.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab.
I20260806 11:36:13.918886 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.195:0
--local_ip_for_outbound_sockets=127.21.14.195
--webserver_interface=127.21.14.195
--webserver_port=0
--tserver_master_addrs=127.21.14.254:40221
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab
--principal=kudu/127.21.14.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:33149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:36:14.040861 22508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:36:14.041177 22508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:36:14.041226 22508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:36:14.045465 22508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:36:14.045660 22508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:36:14.045782 22508 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.195
I20260806 11:36:14.051666 22508 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:33149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab
--principal=kudu/127.21.14.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.14.195
--webserver_port=0
--tserver_master_addrs=127.21.14.254:40221
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22508
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.195
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:36:14.052925 22508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:36:14.054051 22508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:36:14.064939 22514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:14.064958 22515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:14.064951 22517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:14.065253 22508 server_base.cc:1034] running on GCE node
I20260806 11:36:14.065744 22508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:36:14.066514 22508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:36:14.067696 22508 hybrid_clock.cc:648] HybridClock initialized: now 1786016174067651 us; error 59 us; skew 500 ppm
Aug 06 11:36:14 dist-test-slave-7c35 krb5kdc[22117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016174, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:36:14.070914 22508 init.cc:377] Logged in from keytab as kudu/127.21.14.195@KRBTEST.COM (short username kudu)
I20260806 11:36:14.072017 22508 webserver.cc:460] Webserver started at http://127.21.14.195:38073/ using document root <none> and password file <none>
I20260806 11:36:14.072669 22508 fs_manager.cc:359] Metadata directory not provided
I20260806 11:36:14.072741 22508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:36:14.072983 22508 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:36:14.075417 22508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/minicluster-data/ts-2/data/instance:
uuid: "1a225d088b684f60b1851bdf49be843f"
format_stamp: "Formatted at 2026-08-06 11:36:14 on dist-test-slave-7c35"
server_key: "b87aeb9662d47139c9f524a0b021377b"
server_key_iv: "fd166f7433c65c30d116a723bcda3328"
server_key_version: "encryptionkey@0"
I20260806 11:36:14.076098 22508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/minicluster-data/ts-2/wal/instance:
uuid: "1a225d088b684f60b1851bdf49be843f"
format_stamp: "Formatted at 2026-08-06 11:36:14 on dist-test-slave-7c35"
server_key: "b87aeb9662d47139c9f524a0b021377b"
server_key_iv: "fd166f7433c65c30d116a723bcda3328"
server_key_version: "encryptionkey@0"
I20260806 11:36:14.080552 22508 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 11:36:14.083854 22524 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:14.085496 22508 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 11:36:14.085670 22508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/minicluster-data/ts-2/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/minicluster-data/ts-2/wal
uuid: "1a225d088b684f60b1851bdf49be843f"
format_stamp: "Formatted at 2026-08-06 11:36:14 on dist-test-slave-7c35"
server_key: "b87aeb9662d47139c9f524a0b021377b"
server_key_iv: "fd166f7433c65c30d116a723bcda3328"
server_key_version: "encryptionkey@0"
I20260806 11:36:14.085813 22508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:36:14.101706 22508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:36:14.106138 22508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:36:14.106400 22508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:36:14.107223 22508 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:36:14.108233 22508 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:36:14.108289 22508 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:14.108337 22508 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:36:14.108503 22508 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:14.118948 22508 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.195:35677
I20260806 11:36:14.118965 22645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.195:35677 every 8 connection(s)
I20260806 11:36:14.120110 22508 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
I20260806 11:36:14.127252 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 22508
I20260806 11:36:14.127408 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786016169045816-21563-0/minicluster-data/ts-2/wal/instance
I20260806 11:36:14.127758 21563 external_mini_cluster.cc:1428] Setting key 9250c1bc48fe5b13e3df0e8a9a0b1d51
Aug 06 11:36:14 dist-test-slave-7c35 krb5kdc[22117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016174, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.195@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:36:14.137383 22203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:14.122540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:60069 (local address 127.21.14.254:40221)
0806 11:36:14.122730 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:14.122734 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:14.124047 (+ 1313us) server_negotiation.cc:407] Connection header received
0806 11:36:14.125108 (+ 1061us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:14.125111 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:14.125180 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:14.125283 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:14.126703 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:14.127531 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:14.129159 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:14.129408 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:14.129533 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:14.132928 (+ 3395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:14.132932 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:14.132934 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:14.132965 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:14.134839 (+ 1874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:14.135583 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:14.135587 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:14.135589 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:14.135689 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:14.136102 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:14.136106 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:14.136107 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:14.136329 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:14.136451 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:14.137050 (+ 599us) server_negotiation.cc:299] Negotiation successful
0806 11:36:14.137204 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 11:36:14.138487 22648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:14.122931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:40221 (local address 127.21.14.195:60069)
0806 11:36:14.123833 (+ 902us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:14.123886 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:14.124844 (+ 958us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:14.125502 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:14.125511 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:14.125942 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:14.126496 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:14.126513 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:14.127726 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:14.127730 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:14.129000 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:14.129016 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:14.129600 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:14.129605 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:14.129801 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:14.130684 (+ 883us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:14.130703 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:14.132669 (+ 1966us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:14.135086 (+ 2417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:14.135096 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:14.135121 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:14.135460 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:14.135794 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:14.135797 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:14.135799 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:14.135995 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:14.136444 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:14.136453 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:14.136763 (+ 310us) client_negotiation.cc:769] Sending connection context
0806 11:36:14.137004 (+ 241us) client_negotiation.cc:240] Negotiation successful
0806 11:36:14.137408 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":667,"thread_start_us":154,"threads_started":1}
I20260806 11:36:14.140311 22646 heartbeater.cc:347] Connected to a master server at 127.21.14.254:40221
I20260806 11:36:14.140641 22646 heartbeater.cc:464] Registering TS with master...
I20260806 11:36:14.141333 22646 heartbeater.cc:511] Master 127.21.14.254:40221 requested a full tablet report, sending...
I20260806 11:36:14.142596 22162 ts_manager.cc:194] Registered new tserver with Master: 1a225d088b684f60b1851bdf49be843f (127.21.14.195:35677)
I20260806 11:36:14.143230 22162 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.14.195@KRBTEST.COM'} at 127.21.14.195:60069
I20260806 11:36:14.155627 21563 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 11:36:14.169771 22203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:14.160658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43428 (local address 127.21.14.254:40221)
0806 11:36:14.160815 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:14.160820 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:14.160997 (+ 177us) server_negotiation.cc:407] Connection header received
0806 11:36:14.161170 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:14.161173 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:14.161231 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:14.161328 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:14.162284 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:14.162921 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:14.163800 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:14.164075 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:14.164193 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:14.165883 (+ 1690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:14.165886 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:14.165889 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:14.165922 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:14.167721 (+ 1799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:14.168291 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:14.168295 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:14.168296 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:14.168426 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:14.168759 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:14.168762 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:14.168763 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:14.169045 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:14.169243 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:14.169497 (+ 254us) server_negotiation.cc:299] Negotiation successful
0806 11:36:14.169622 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 11:36:14.177325 22162 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43428:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 11:36:14.181382 22162 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 11:36:14.201181 22663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:14.194792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57572 (local address 127.21.14.194:46161)
0806 11:36:14.195617 (+ 825us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:14.195629 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:14.196290 (+ 661us) server_negotiation.cc:407] Connection header received
0806 11:36:14.196385 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:14.196389 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:14.196531 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:14.196684 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:36:14.196881 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:14.197674 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:14.199475 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:14.200296 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:14.200423 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:14.200644 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:14.200808 (+ 164us) server_negotiation.cc:299] Negotiation successful
0806 11:36:14.200999 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":721,"thread_start_us":188,"threads_started":1}
I20260806 11:36:14.201166 22658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:14.194589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.194:46161 (local address 127.0.0.1:57572)
0806 11:36:14.196059 (+ 1470us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:14.196082 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:14.196207 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:14.196663 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:14.196666 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:14.196683 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:36:14.196784 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:14.196791 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:14.197829 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:14.197833 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:14.199340 (+ 1507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:14.199352 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:14.200464 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:14.200467 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:14.200571 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:14.200607 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 11:36:14.200764 (+ 157us) client_negotiation.cc:240] Negotiation successful
0806 11:36:14.200994 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1324,"mutex_wait_us":85,"spinlock_wait_cycles":768,"thread_start_us":102,"threads_started":1}
I20260806 11:36:14.201905 22659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:14.193764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.193:39883 (local address 127.0.0.1:46054)
0806 11:36:14.194540 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:14.194607 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:14.194774 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:14.195738 (+ 964us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:14.195745 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:14.195772 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:36:14.195971 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:14.195980 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:14.197676 (+ 1696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:14.197679 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:14.199198 (+ 1519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:14.199209 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:14.201477 (+ 2268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:14.201481 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:14.201590 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:14.201622 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 11:36:14.201690 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 11:36:14.201768 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"mutex_wait_us":204,"thread_start_us":114,"threads_started":1}
I20260806 11:36:14.202562 22661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:14.193814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.195:35677 (local address 127.0.0.1:51886)
0806 11:36:14.195090 (+ 1276us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:14.195105 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:14.195191 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:14.196126 (+ 935us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:14.196128 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:14.196144 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:36:14.196230 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:14.196238 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:14.197637 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:14.197640 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:14.199340 (+ 1700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:14.199352 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:14.201853 (+ 2501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:14.201856 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:14.201934 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:14.201951 (+ 17us) client_negotiation.cc:769] Sending connection context
0806 11:36:14.202171 (+ 220us) client_negotiation.cc:240] Negotiation successful
0806 11:36:14.202232 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1217,"mutex_wait_us":103,"thread_start_us":113,"threads_started":1}
I20260806 11:36:14.203145 22660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:14.193868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46054 (local address 127.21.14.193:39883)
0806 11:36:14.194325 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:14.194334 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:14.194925 (+ 591us) server_negotiation.cc:407] Connection header received
0806 11:36:14.195122 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:14.195130 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:14.195567 (+ 437us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:14.195732 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:36:14.196144 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:14.197494 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:14.199368 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:14.200296 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:14.200423 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:14.202747 (+ 2324us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:14.202824 (+ 77us) server_negotiation.cc:299] Negotiation successful
0806 11:36:14.202976 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":124,"threads_started":1}
I20260806 11:36:14.203156 22662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:14.194647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51886 (local address 127.21.14.195:35677)
0806 11:36:14.195580 (+ 933us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:14.195594 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:14.195620 (+ 26us) server_negotiation.cc:407] Connection header received
0806 11:36:14.195717 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:14.195722 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:14.195955 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:14.196116 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:36:14.196357 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:14.197494 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:14.200715 (+ 3221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:14.201600 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:14.201687 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:14.201797 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:14.202908 (+ 1111us) server_negotiation.cc:299] Negotiation successful
0806 11:36:14.203010 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":810,"thread_start_us":214,"threads_started":1}
I20260806 11:36:14.205392 22434 tablet_service.cc:1467] Processing CreateTablet for tablet a0e300ee8e9749dc887664e2a570736f (DEFAULT_TABLE table=test-table [id=959c89cf58f84f29a5048dc94de82752]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:36:14.206353 22292 tablet_service.cc:1467] Processing CreateTablet for tablet a0e300ee8e9749dc887664e2a570736f (DEFAULT_TABLE table=test-table [id=959c89cf58f84f29a5048dc94de82752]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:36:14.206984 22434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0e300ee8e9749dc887664e2a570736f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:14.207947 22292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0e300ee8e9749dc887664e2a570736f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:14.207602 22575 tablet_service.cc:1467] Processing CreateTablet for tablet a0e300ee8e9749dc887664e2a570736f (DEFAULT_TABLE table=test-table [id=959c89cf58f84f29a5048dc94de82752]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:36:14.208827 22575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0e300ee8e9749dc887664e2a570736f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:14.215078 22664 tablet_bootstrap.cc:492] T a0e300ee8e9749dc887664e2a570736f P 08ad0378c4ed4dbbac3df4e736e9354a: Bootstrap starting.
I20260806 11:36:14.216677 22666 tablet_bootstrap.cc:492] T a0e300ee8e9749dc887664e2a570736f P 1a225d088b684f60b1851bdf49be843f: Bootstrap starting.
I20260806 11:36:14.217810 22665 tablet_bootstrap.cc:492] T a0e300ee8e9749dc887664e2a570736f P db49f67cc2c14f3ba007fabc0acb7b8d: Bootstrap starting.
I20260806 11:36:14.217931 22664 tablet_bootstrap.cc:654] T a0e300ee8e9749dc887664e2a570736f P 08ad0378c4ed4dbbac3df4e736e9354a: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:14.219261 22664 log.cc:824] T a0e300ee8e9749dc887664e2a570736f P 08ad0378c4ed4dbbac3df4e736e9354a: Log is configured to *not* fsync() on all Append() calls
I20260806 11:36:14.219321 22666 tablet_bootstrap.cc:654] T a0e300ee8e9749dc887664e2a570736f P 1a225d088b684f60b1851bdf49be843f: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:14.220316 22665 tablet_bootstrap.cc:654] T a0e300ee8e9749dc887664e2a570736f P db49f67cc2c14f3ba007fabc0acb7b8d: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:14.221056 22666 log.cc:824] T a0e300ee8e9749dc887664e2a570736f P 1a225d088b684f60b1851bdf49be843f: Log is configured to *not* fsync() on all Append() calls
I20260806 11:36:14.221391 22664 tablet_bootstrap.cc:492] T a0e300ee8e9749dc887664e2a570736f P 08ad0378c4ed4dbbac3df4e736e9354a: No bootstrap required, opened a new log
I20260806 11:36:14.221623 22664 ts_tablet_manager.cc:1397] T a0e300ee8e9749dc887664e2a570736f P 08ad0378c4ed4dbbac3df4e736e9354a: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260806 11:36:14.221709 22665 log.cc:824] T a0e300ee8e9749dc887664e2a570736f P db49f67cc2c14f3ba007fabc0acb7b8d: Log is configured to *not* fsync() on all Append() calls
I20260806 11:36:14.223376 22666 tablet_bootstrap.cc:492] T a0e300ee8e9749dc887664e2a570736f P 1a225d088b684f60b1851bdf49be843f: No bootstrap required, opened a new log
I20260806 11:36:14.223613 22666 ts_tablet_manager.cc:1397] T a0e300ee8e9749dc887664e2a570736f P 1a225d088b684f60b1851bdf49be843f: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260806 11:36:14.224045 22665 tablet_bootstrap.cc:492] T a0e300ee8e9749dc887664e2a570736f P db49f67cc2c14f3ba007fabc0acb7b8d: No bootstrap required, opened a new log
I20260806 11:36:14.224246 22665 ts_tablet_manager.cc:1397] T a0e300ee8e9749dc887664e2a570736f P db49f67cc2c14f3ba007fabc0acb7b8d: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 11:36:14.226743 22664 raft_consensus.cc:348] T a0e300ee8e9749dc887664e2a570736f P 08ad0378c4ed4dbbac3df4e736e9354a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08ad0378c4ed4dbbac3df4e736e9354a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 39883 } } peers { permanent_uuid: "1a225d088b684f60b1851bdf49be843f" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 35677 } } peers { permanent_uuid: "db49f67cc2c14f3ba007fabc0acb7b8d" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 46161 } }
I20260806 11:36:14.227097 22664 raft_consensus.cc:374] T a0e300ee8e9749dc887664e2a570736f P 08ad0378c4ed4dbbac3df4e736e9354a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:14.227169 22664 raft_consensus.cc:729] T a0e300ee8e9749dc887664e2a570736f P 08ad0378c4ed4dbbac3df4e736e9354a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08ad0378c4ed4dbbac3df4e736e9354a, State: Initialized, Role: FOLLOWER
I20260806 11:36:14.227658 22664 consensus_queue.cc:260] T a0e300ee8e9749dc887664e2a570736f P 08ad0378c4ed4dbbac3df4e736e9354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08ad0378c4ed4dbbac3df4e736e9354a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 39883 } } peers { permanent_uuid: "1a225d088b684f60b1851bdf49be843f" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 35677 } } peers { permanent_uuid: "db49f67cc2c14f3ba007fabc0acb7b8d" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 46161 } }
I20260806 11:36:14.228219 22666 raft_consensus.cc:348] T a0e300ee8e9749dc887664e2a570736f P 1a225d088b684f60b1851bdf49be843f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08ad0378c4ed4dbbac3df4e736e9354a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 39883 } } peers { permanent_uuid: "1a225d088b684f60b1851bdf49be843f" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 35677 } } peers { permanent_uuid: "db49f67cc2c14f3ba007fabc0acb7b8d" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 46161 } }
I20260806 11:36:14.228513 22362 heartbeater.cc:503] Master 127.21.14.254:40221 was elected leader, sending a full tablet report...
I20260806 11:36:14.228533 22666 raft_consensus.cc:374] T a0e300ee8e9749dc887664e2a570736f P 1a225d088b684f60b1851bdf49be843f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:14.228588 22666 raft_consensus.cc:729] T a0e300ee8e9749dc887664e2a570736f P 1a225d088b684f60b1851bdf49be843f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a225d088b684f60b1851bdf49be843f, State: Initialized, Role: FOLLOWER
I20260806 11:36:14.228933 22664 ts_tablet_manager.cc:1428] T a0e300ee8e9749dc887664e2a570736f P 08ad0378c4ed4dbbac3df4e736e9354a: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260806 11:36:14.229070 22666 consensus_queue.cc:260] T a0e300ee8e9749dc887664e2a570736f P 1a225d088b684f60b1851bdf49be843f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08ad0378c4ed4dbbac3df4e736e9354a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 39883 } } peers { permanent_uuid: "1a225d088b684f60b1851bdf49be843f" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 35677 } } peers { permanent_uuid: "db49f67cc2c14f3ba007fabc0acb7b8d" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 46161 } }
I20260806 11:36:14.229212 22665 raft_consensus.cc:348] T a0e300ee8e9749dc887664e2a570736f P db49f67cc2c14f3ba007fabc0acb7b8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08ad0378c4ed4dbbac3df4e736e9354a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 39883 } } peers { permanent_uuid: "1a225d088b684f60b1851bdf49be843f" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 35677 } } peers { permanent_uuid: "db49f67cc2c14f3ba007fabc0acb7b8d" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 46161 } }
I20260806 11:36:14.229489 22665 raft_consensus.cc:374] T a0e300ee8e9749dc887664e2a570736f P db49f67cc2c14f3ba007fabc0acb7b8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:14.229554 22665 raft_consensus.cc:729] T a0e300ee8e9749dc887664e2a570736f P db49f67cc2c14f3ba007fabc0acb7b8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db49f67cc2c14f3ba007fabc0acb7b8d, State: Initialized, Role: FOLLOWER
I20260806 11:36:14.229846 22646 heartbeater.cc:503] Master 127.21.14.254:40221 was elected leader, sending a full tablet report...
I20260806 11:36:14.230039 22665 consensus_queue.cc:260] T a0e300ee8e9749dc887664e2a570736f P db49f67cc2c14f3ba007fabc0acb7b8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08ad0378c4ed4dbbac3df4e736e9354a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 39883 } } peers { permanent_uuid: "1a225d088b684f60b1851bdf49be843f" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 35677 } } peers { permanent_uuid: "db49f67cc2c14f3ba007fabc0acb7b8d" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 46161 } }
I20260806 11:36:14.230275 22666 ts_tablet_manager.cc:1428] T a0e300ee8e9749dc887664e2a570736f P 1a225d088b684f60b1851bdf49be843f: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260806 11:36:14.230901 22502 heartbeater.cc:503] Master 127.21.14.254:40221 was elected leader, sending a full tablet report...
I20260806 11:36:14.231071 22665 ts_tablet_manager.cc:1428] T a0e300ee8e9749dc887664e2a570736f P db49f67cc2c14f3ba007fabc0acb7b8d: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260806 11:36:14.358767 22671 raft_consensus.cc:482] T a0e300ee8e9749dc887664e2a570736f P 1a225d088b684f60b1851bdf49be843f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 11:36:14.359069 22671 raft_consensus.cc:504] T a0e300ee8e9749dc887664e2a570736f P 1a225d088b684f60b1851bdf49be843f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08ad0378c4ed4dbbac3df4e736e9354a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 39883 } } peers { permanent_uuid: "1a225d088b684f60b1851bdf49be843f" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 35677 } } peers { permanent_uuid: "db49f67cc2c14f3ba007fabc0acb7b8d" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 46161 } }
I20260806 11:36:14.360334 22671 leader_election.cc:288] T a0e300ee8e9749dc887664e2a570736f P 1a225d088b684f60b1851bdf49be843f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 08ad0378c4ed4dbbac3df4e736e9354a (127.21.14.193:39883), db49f67cc2c14f3ba007fabc0acb7b8d (127.21.14.194:46161)
I20260806 11:36:14.365641 22663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:14.361087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:56749 (local address 127.21.14.194:46161)
0806 11:36:14.361269 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:14.361276 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:14.361461 (+ 185us) server_negotiation.cc:407] Connection header received
0806 11:36:14.361566 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:14.361568 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:14.361645 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:14.361741 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:36:14.362043 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:14.362718 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:14.364168 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:14.364805 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:14.364946 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:14.364994 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:14.365400 (+ 406us) server_negotiation.cc:299] Negotiation successful
0806 11:36:14.365493 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 11:36:14.365641 22660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:14.360828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:35737 (local address 127.21.14.193:39883)
0806 11:36:14.361026 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:14.361031 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:14.361050 (+ 19us) server_negotiation.cc:407] Connection header received
0806 11:36:14.361153 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:14.361158 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:14.361214 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:14.361307 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:36:14.361645 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:14.362465 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:14.364062 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:14.364774 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:14.364929 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:14.364994 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:14.365400 (+ 406us) server_negotiation.cc:299] Negotiation successful
0806 11:36:14.365515 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118}
I20260806 11:36:14.365758 22648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:14.360652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.193:39883 (local address 127.21.14.195:35737)
0806 11:36:14.360871 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:14.360904 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:14.361039 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:14.361341 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:14.361344 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:14.361370 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:36:14.361518 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:14.361528 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:14.362586 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:14.362588 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:14.363897 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:14.363910 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:14.364956 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:14.364961 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:14.365201 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:14.365267 (+ 66us) client_negotiation.cc:769] Sending connection context
0806 11:36:14.365360 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 11:36:14.365580 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72,"mutex_wait_us":187}
I20260806 11:36:14.365767 22673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:14.360941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.194:46161 (local address 127.21.14.195:56749)
0806 11:36:14.361353 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:14.361366 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:14.361465 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:14.361768 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:14.361772 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:14.361793 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:36:14.361913 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:14.361920 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:14.362823 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:14.362826 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:14.364021 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:14.364030 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:14.364956 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:14.364961 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:14.365196 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:14.365271 (+ 75us) client_negotiation.cc:769] Sending connection context
0806 11:36:14.365360 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 11:36:14.365580 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"mutex_wait_us":111,"thread_start_us":128,"threads_started":1}
I20260806 11:36:14.366639 22314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0e300ee8e9749dc887664e2a570736f" candidate_uuid: "1a225d088b684f60b1851bdf49be843f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08ad0378c4ed4dbbac3df4e736e9354a" is_pre_election: true
I20260806 11:36:14.366644 22455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0e300ee8e9749dc887664e2a570736f" candidate_uuid: "1a225d088b684f60b1851bdf49be843f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db49f67cc2c14f3ba007fabc0acb7b8d" is_pre_election: true
I20260806 11:36:14.367143 22455 raft_consensus.cc:2454] T a0e300ee8e9749dc887664e2a570736f P db49f67cc2c14f3ba007fabc0acb7b8d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a225d088b684f60b1851bdf49be843f in term 0.
I20260806 11:36:14.367847 22525 leader_election.cc:302] T a0e300ee8e9749dc887664e2a570736f P 1a225d088b684f60b1851bdf49be843f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a225d088b684f60b1851bdf49be843f, db49f67cc2c14f3ba007fabc0acb7b8d; no voters:
I20260806 11:36:14.368216 22671 raft_consensus.cc:2781] T a0e300ee8e9749dc887664e2a570736f P 1a225d088b684f60b1851bdf49be843f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 11:36:14.368306 22671 raft_consensus.cc:482] T a0e300ee8e9749dc887664e2a570736f P 1a225d088b684f60b1851bdf49be843f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 11:36:14.368338 22671 raft_consensus.cc:3037] T a0e300ee8e9749dc887664e2a570736f P 1a225d088b684f60b1851bdf49be843f [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:36:14.369181 22314 raft_consensus.cc:2454] T a0e300ee8e9749dc887664e2a570736f P 08ad0378c4ed4dbbac3df4e736e9354a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a225d088b684f60b1851bdf49be843f in term 0.
I20260806 11:36:14.369735 22671 raft_consensus.cc:504] T a0e300ee8e9749dc887664e2a570736f P 1a225d088b684f60b1851bdf49be843f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08ad0378c4ed4dbbac3df4e736e9354a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 39883 } } peers { permanent_uuid: "1a225d088b684f60b1851bdf49be843f" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 35677 } } peers { permanent_uuid: "db49f67cc2c14f3ba007fabc0acb7b8d" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 46161 } }
I20260806 11:36:14.370186 22671 leader_election.cc:288] T a0e300ee8e9749dc887664e2a570736f P 1a225d088b684f60b1851bdf49be843f [CANDIDATE]: Term 1 election: Requested vote from peers 08ad0378c4ed4dbbac3df4e736e9354a (127.21.14.193:39883), db49f67cc2c14f3ba007fabc0acb7b8d (127.21.14.194:46161)
I20260806 11:36:14.370700 22314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0e300ee8e9749dc887664e2a570736f" candidate_uuid: "1a225d088b684f60b1851bdf49be843f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08ad0378c4ed4dbbac3df4e736e9354a"
I20260806 11:36:14.370700 22455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0e300ee8e9749dc887664e2a570736f" candidate_uuid: "1a225d088b684f60b1851bdf49be843f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db49f67cc2c14f3ba007fabc0acb7b8d"
I20260806 11:36:14.370896 22314 raft_consensus.cc:3037] T a0e300ee8e9749dc887664e2a570736f P 08ad0378c4ed4dbbac3df4e736e9354a [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:36:14.370894 22455 raft_consensus.cc:3037] T a0e300ee8e9749dc887664e2a570736f P db49f67cc2c14f3ba007fabc0acb7b8d [term 0 FOLLOWER]: Advancing to term 1
W20260806 11:36:14.371183 22647 tablet.cc:2369] T a0e300ee8e9749dc887664e2a570736f P 1a225d088b684f60b1851bdf49be843f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 11:36:14.372357 22455 raft_consensus.cc:2454] T a0e300ee8e9749dc887664e2a570736f P db49f67cc2c14f3ba007fabc0acb7b8d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a225d088b684f60b1851bdf49be843f in term 1.
I20260806 11:36:14.372352 22314 raft_consensus.cc:2454] T a0e300ee8e9749dc887664e2a570736f P 08ad0378c4ed4dbbac3df4e736e9354a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a225d088b684f60b1851bdf49be843f in term 1.
I20260806 11:36:14.372745 22525 leader_election.cc:302] T a0e300ee8e9749dc887664e2a570736f P 1a225d088b684f60b1851bdf49be843f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a225d088b684f60b1851bdf49be843f, db49f67cc2c14f3ba007fabc0acb7b8d; no voters:
I20260806 11:36:14.372967 22671 raft_consensus.cc:2781] T a0e300ee8e9749dc887664e2a570736f P 1a225d088b684f60b1851bdf49be843f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:36:14.373219 22671 raft_consensus.cc:686] T a0e300ee8e9749dc887664e2a570736f P 1a225d088b684f60b1851bdf49be843f [term 1 LEADER]: Becoming Leader. State: Replica: 1a225d088b684f60b1851bdf49be843f, State: Running, Role: LEADER
I20260806 11:36:14.373620 22671 consensus_queue.cc:237] T a0e300ee8e9749dc887664e2a570736f P 1a225d088b684f60b1851bdf49be843f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08ad0378c4ed4dbbac3df4e736e9354a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 39883 } } peers { permanent_uuid: "1a225d088b684f60b1851bdf49be843f" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 35677 } } peers { permanent_uuid: "db49f67cc2c14f3ba007fabc0acb7b8d" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 46161 } }
W20260806 11:36:14.377358 22363 tablet.cc:2369] T a0e300ee8e9749dc887664e2a570736f P 08ad0378c4ed4dbbac3df4e736e9354a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 11:36:14.378638 22162 catalog_manager.cc:5495] T a0e300ee8e9749dc887664e2a570736f P 1a225d088b684f60b1851bdf49be843f reported cstate change: term changed from 0 to 1, leader changed from <none> to 1a225d088b684f60b1851bdf49be843f (127.21.14.195). New cstate: current_term: 1 leader_uuid: "1a225d088b684f60b1851bdf49be843f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08ad0378c4ed4dbbac3df4e736e9354a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 39883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a225d088b684f60b1851bdf49be843f" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 35677 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db49f67cc2c14f3ba007fabc0acb7b8d" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 46161 } health_report { overall_health: UNKNOWN } } }
W20260806 11:36:14.382802 22503 tablet.cc:2369] T a0e300ee8e9749dc887664e2a570736f P db49f67cc2c14f3ba007fabc0acb7b8d: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 11:36:14 dist-test-slave-7c35 krb5kdc[22117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016172, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.14.193@KRBTEST.COM
I20260806 11:36:14.398232 22660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:14.388110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46068 (local address 127.21.14.193:39883)
0806 11:36:14.388276 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:14.388281 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:14.388485 (+ 204us) server_negotiation.cc:407] Connection header received
0806 11:36:14.388675 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:14.388679 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:14.388726 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:14.388815 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:14.389571 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:14.390208 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:14.391225 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:14.391473 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:14.391583 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:14.394195 (+ 2612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:14.394212 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:14.394231 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:14.394265 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:14.396193 (+ 1928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:14.396744 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:14.396750 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:14.396751 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:14.396850 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:14.397174 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:14.397177 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:14.397178 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:14.397460 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:14.397692 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:14.397896 (+ 204us) server_negotiation.cc:299] Negotiation successful
0806 11:36:14.398037 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
Aug 06 11:36:14 dist-test-slave-7c35 krb5kdc[22117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016174, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 11:36:14 dist-test-slave-7c35 krb5kdc[22117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016174, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.14.193@KRBTEST.COM
I20260806 11:36:14.429535 22660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:14.420541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46074 (local address 127.21.14.193:39883)
0806 11:36:14.420722 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:14.420726 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:14.420747 (+ 21us) server_negotiation.cc:407] Connection header received
0806 11:36:14.420913 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:14.420916 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:14.420968 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:14.421052 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:14.421862 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:14.422433 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:14.423363 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:14.423611 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:14.423709 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:14.426175 (+ 2466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:14.426178 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:14.426180 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:14.426209 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:14.427776 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:14.428227 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:14.428230 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:14.428232 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:14.428317 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:14.428588 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:14.428592 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:14.428594 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:14.428775 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:14.428907 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:14.429223 (+ 316us) server_negotiation.cc:299] Negotiation successful
0806 11:36:14.429360 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
Aug 06 11:36:14 dist-test-slave-7c35 krb5kdc[22117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016174, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 11:36:14 dist-test-slave-7c35 krb5kdc[22117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016174, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.14.193@KRBTEST.COM
I20260806 11:36:14.461428 22660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:14.451001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46084 (local address 127.21.14.193:39883)
0806 11:36:14.451213 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:14.451218 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:14.451822 (+ 604us) server_negotiation.cc:407] Connection header received
0806 11:36:14.452023 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:14.452027 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:14.452087 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:14.452136 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:14.452985 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:14.453517 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:14.454496 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:14.454740 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:14.454833 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:14.457964 (+ 3131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:14.457968 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:14.457970 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:14.458011 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:14.459557 (+ 1546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:14.460076 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:14.460079 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:14.460080 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:14.460167 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:14.460550 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:14.460553 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:14.460554 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:14.460738 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:14.460841 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:14.461139 (+ 298us) server_negotiation.cc:299] Negotiation successful
0806 11:36:14.461296 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
W20260806 11:36:14.463457 22273 tablet_service.cc:2908] Rejecting scan request for tablet a0e300ee8e9749dc887664e2a570736f: Uninitialized: clean time has not yet been initialized
I20260806 11:36:14.466310 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 22228
I20260806 11:36:14.474591 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 22367
I20260806 11:36:14.482897 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 22508
I20260806 11:36:14.490983 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 22130
2026-08-06T11:36:14Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2852 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 11:36:14 dist-test-slave-7c35 krb5kdc[22690](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 11:36:14 dist-test-slave-7c35 krb5kdc[22690](info): set up 2 sockets
Aug 06 11:36:14 dist-test-slave-7c35 krb5kdc[22690](info): commencing operation
krb5kdc: starting...
W20260806 11:36:15.596644 21563 mini_kdc.cc:121] Time spent starting KDC: real 1.080s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Aug 06 11:36:15 dist-test-slave-7c35 krb5kdc[22690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016175, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T11:36:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T11:36:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.14.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab.
Entry for principal kudu/127.21.14.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab.
I20260806 11:36:15.747355 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:40925
--webserver_interface=127.21.14.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab
--principal=kudu/127.21.14.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:37439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.14.254:40925
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:36:15.884266 22703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:36:15.884658 22703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:36:15.884752 22703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:36:15.889619 22703 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 11:36:15.889721 22703 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 11:36:15.889740 22703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:36:15.889758 22703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:36:15.889776 22703 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 11:36:15.889816 22703 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 11:36:15.896203 22703 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:37439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.14.254:40925
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab
--principal=kudu/127.21.14.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:40925
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.14.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22703
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:36:15.897473 22703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:36:15.898612 22703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:36:15.908550 22711 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:15.908543 22708 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:15.908512 22709 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:15.908939 22703 server_base.cc:1034] running on GCE node
I20260806 11:36:15.909659 22703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:36:15.911056 22703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:36:15.912232 22703 hybrid_clock.cc:648] HybridClock initialized: now 1786016175912195 us; error 55 us; skew 500 ppm
Aug 06 11:36:15 dist-test-slave-7c35 krb5kdc[22690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016175, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:36:15.915433 22703 init.cc:377] Logged in from keytab as kudu/127.21.14.254@KRBTEST.COM (short username kudu)
I20260806 11:36:15.916580 22703 webserver.cc:460] Webserver started at http://127.21.14.254:44521/ using document root <none> and password file <none>
I20260806 11:36:15.917234 22703 fs_manager.cc:359] Metadata directory not provided
I20260806 11:36:15.917307 22703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:36:15.917567 22703 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:36:15.919929 22703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/master-0/data/instance:
uuid: "0569d29f335e4760ad00e3096cef03fc"
format_stamp: "Formatted at 2026-08-06 11:36:15 on dist-test-slave-7c35"
server_key: "6778577c868c10f736328729c8d456ce"
server_key_iv: "24722c8a572857ffa18ca2e0e652af35"
server_key_version: "encryptionkey@0"
I20260806 11:36:15.920593 22703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/master-0/wal/instance:
uuid: "0569d29f335e4760ad00e3096cef03fc"
format_stamp: "Formatted at 2026-08-06 11:36:15 on dist-test-slave-7c35"
server_key: "6778577c868c10f736328729c8d456ce"
server_key_iv: "24722c8a572857ffa18ca2e0e652af35"
server_key_version: "encryptionkey@0"
I20260806 11:36:15.924988 22703 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 11:36:15.927868 22719 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:15.929365 22703 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 11:36:15.929533 22703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/master-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/master-0/wal
uuid: "0569d29f335e4760ad00e3096cef03fc"
format_stamp: "Formatted at 2026-08-06 11:36:15 on dist-test-slave-7c35"
server_key: "6778577c868c10f736328729c8d456ce"
server_key_iv: "24722c8a572857ffa18ca2e0e652af35"
server_key_version: "encryptionkey@0"
I20260806 11:36:15.929673 22703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:36:15.961980 22703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:36:15.965816 22703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:36:15.966078 22703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:36:15.975808 22703 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.254:40925
I20260806 11:36:15.975838 22785 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.254:40925 every 8 connection(s)
I20260806 11:36:15.976966 22703 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
I20260806 11:36:15.980183 22786 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:15.986677 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 22703
I20260806 11:36:15.986826 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/master-0/wal/instance
I20260806 11:36:15.986857 22786 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0569d29f335e4760ad00e3096cef03fc: Bootstrap starting.
I20260806 11:36:15.987169 21563 external_mini_cluster.cc:1428] Setting key 4d527d56aca63add1c18ad03e2fe7ce4
I20260806 11:36:15.989928 22786 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0569d29f335e4760ad00e3096cef03fc: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:15.990795 22786 log.cc:824] T 00000000000000000000000000000000 P 0569d29f335e4760ad00e3096cef03fc: Log is configured to *not* fsync() on all Append() calls
I20260806 11:36:15.993028 22786 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0569d29f335e4760ad00e3096cef03fc: No bootstrap required, opened a new log
Aug 06 11:36:15 dist-test-slave-7c35 krb5kdc[22690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016175, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:36:15.997223 22786 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0569d29f335e4760ad00e3096cef03fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0569d29f335e4760ad00e3096cef03fc" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 40925 } }
I20260806 11:36:15.997555 22786 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0569d29f335e4760ad00e3096cef03fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:15.997637 22786 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0569d29f335e4760ad00e3096cef03fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0569d29f335e4760ad00e3096cef03fc, State: Initialized, Role: FOLLOWER
I20260806 11:36:15.998143 22786 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0569d29f335e4760ad00e3096cef03fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0569d29f335e4760ad00e3096cef03fc" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 40925 } }
I20260806 11:36:15.998314 22786 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0569d29f335e4760ad00e3096cef03fc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 11:36:15.998394 22786 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0569d29f335e4760ad00e3096cef03fc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 11:36:15.998548 22786 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0569d29f335e4760ad00e3096cef03fc [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:36:15.999651 22786 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0569d29f335e4760ad00e3096cef03fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0569d29f335e4760ad00e3096cef03fc" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 40925 } }
I20260806 11:36:16.000061 22786 leader_election.cc:302] T 00000000000000000000000000000000 P 0569d29f335e4760ad00e3096cef03fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0569d29f335e4760ad00e3096cef03fc; no voters:
I20260806 11:36:16.000430 22786 leader_election.cc:288] T 00000000000000000000000000000000 P 0569d29f335e4760ad00e3096cef03fc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 11:36:16.000578 22791 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0569d29f335e4760ad00e3096cef03fc [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:36:16.000919 22791 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0569d29f335e4760ad00e3096cef03fc [term 1 LEADER]: Becoming Leader. State: Replica: 0569d29f335e4760ad00e3096cef03fc, State: Running, Role: LEADER
I20260806 11:36:16.001271 22791 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0569d29f335e4760ad00e3096cef03fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0569d29f335e4760ad00e3096cef03fc" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 40925 } }
I20260806 11:36:16.002089 22786 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0569d29f335e4760ad00e3096cef03fc [sys.catalog]: configured and running, proceeding with master startup.
I20260806 11:36:16.003340 22792 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0569d29f335e4760ad00e3096cef03fc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0569d29f335e4760ad00e3096cef03fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0569d29f335e4760ad00e3096cef03fc" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 40925 } } }
I20260806 11:36:16.003517 22792 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0569d29f335e4760ad00e3096cef03fc [sys.catalog]: This master's current role is: LEADER
I20260806 11:36:16.004714 22789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:15.988903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50344 (local address 127.21.14.254:40925)
0806 11:36:15.989484 (+ 581us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:15.989501 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:15.989547 (+ 46us) server_negotiation.cc:407] Connection header received
0806 11:36:15.990379 (+ 832us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:15.990402 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:15.990792 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:15.991268 (+ 476us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:15.992137 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:15.993235 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:15.994172 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:15.994701 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:15.994939 (+ 238us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:15.997926 (+ 2987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:15.997933 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:15.997956 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:15.998001 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:16.000073 (+ 2072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:16.000645 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:16.000653 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:16.000661 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:16.000843 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:16.001212 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:16.001215 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:16.001217 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:16.001794 (+ 577us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:16.002018 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:16.003092 (+ 1074us) server_negotiation.cc:299] Negotiation successful
0806 11:36:16.003505 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":183,"threads_started":1}
I20260806 11:36:16.006222 22793 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0569d29f335e4760ad00e3096cef03fc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0569d29f335e4760ad00e3096cef03fc. Latest consensus state: current_term: 1 leader_uuid: "0569d29f335e4760ad00e3096cef03fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0569d29f335e4760ad00e3096cef03fc" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 40925 } } }
I20260806 11:36:16.006363 22793 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0569d29f335e4760ad00e3096cef03fc [sys.catalog]: This master's current role is: LEADER
W20260806 11:36:16.007911 22807 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0569d29f335e4760ad00e3096cef03fc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 11:36:16.007995 22807 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 11:36:16.008076 22800 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 11:36:16.008903 22800 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 11:36:16.014763 22800 catalog_manager.cc:1347] Generated new cluster ID: ae7d08dfeeff45228bb1c2d2965115e8
I20260806 11:36:16.014894 22800 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 11:36:16.038259 22800 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 11:36:16.039561 22800 catalog_manager.cc:1504] Loading token signing keys...
I20260806 11:36:16.044672 22800 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0569d29f335e4760ad00e3096cef03fc: Generated new TSK 0
I20260806 11:36:16.045630 22800 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.14.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab.
Entry for principal kudu/127.21.14.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab.
I20260806 11:36:16.083425 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.193:0
--local_ip_for_outbound_sockets=127.21.14.193
--webserver_interface=127.21.14.193
--webserver_port=0
--tserver_master_addrs=127.21.14.254:40925
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab
--principal=kudu/127.21.14.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:37439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:36:16.218292 22813 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:36:16.218639 22813 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:36:16.218690 22813 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:36:16.223483 22813 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:36:16.223582 22813 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:36:16.223626 22813 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 11:36:16.223695 22813 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.193
I20260806 11:36:16.230281 22813 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:37439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab
--principal=kudu/127.21.14.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.14.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.14.254:40925
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22813
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.193
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:36:16.231778 22813 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:36:16.233027 22813 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:36:16.245249 22819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:16.245254 22818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:16.245590 22813 server_base.cc:1034] running on GCE node
W20260806 11:36:16.245249 22821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:16.246279 22813 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:36:16.247133 22813 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:36:16.248347 22813 hybrid_clock.cc:648] HybridClock initialized: now 1786016176248326 us; error 38 us; skew 500 ppm
Aug 06 11:36:16 dist-test-slave-7c35 krb5kdc[22690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016176, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:36:16.251689 22813 init.cc:377] Logged in from keytab as kudu/127.21.14.193@KRBTEST.COM (short username kudu)
I20260806 11:36:16.252802 22813 webserver.cc:460] Webserver started at http://127.21.14.193:43453/ using document root <none> and password file <none>
I20260806 11:36:16.253471 22813 fs_manager.cc:359] Metadata directory not provided
I20260806 11:36:16.253571 22813 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:36:16.253823 22813 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:36:16.256285 22813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/ts-0/data/instance:
uuid: "5b2c000f19594cfa8696548a481014b7"
format_stamp: "Formatted at 2026-08-06 11:36:16 on dist-test-slave-7c35"
server_key: "bf26458dd1c2da8630fe23eb28e951fb"
server_key_iv: "dc17270cef7c122f78fc704a94466baf"
server_key_version: "encryptionkey@0"
I20260806 11:36:16.257030 22813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/ts-0/wal/instance:
uuid: "5b2c000f19594cfa8696548a481014b7"
format_stamp: "Formatted at 2026-08-06 11:36:16 on dist-test-slave-7c35"
server_key: "bf26458dd1c2da8630fe23eb28e951fb"
server_key_iv: "dc17270cef7c122f78fc704a94466baf"
server_key_version: "encryptionkey@0"
I20260806 11:36:16.262059 22813 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 11:36:16.265053 22829 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:16.266533 22813 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 11:36:16.266695 22813 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/ts-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/ts-0/wal
uuid: "5b2c000f19594cfa8696548a481014b7"
format_stamp: "Formatted at 2026-08-06 11:36:16 on dist-test-slave-7c35"
server_key: "bf26458dd1c2da8630fe23eb28e951fb"
server_key_iv: "dc17270cef7c122f78fc704a94466baf"
server_key_version: "encryptionkey@0"
I20260806 11:36:16.266829 22813 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:36:16.285722 22813 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:36:16.289448 22813 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:36:16.289697 22813 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:36:16.291464 22813 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:36:16.291541 22813 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:16.291615 22813 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:36:16.291662 22813 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 11:36:16 dist-test-slave-7c35 krb5kdc[22690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016176, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.193@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:36:16.303936 22813 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.193:40535
I20260806 11:36:16.305202 22813 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
I20260806 11:36:16.305802 22949 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.193:40535 every 8 connection(s)
I20260806 11:36:16.307257 22789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:16.292485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:35379 (local address 127.21.14.254:40925)
0806 11:36:16.292676 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:16.292681 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:16.293809 (+ 1128us) server_negotiation.cc:407] Connection header received
0806 11:36:16.294954 (+ 1145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:16.294958 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:16.295047 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:16.295154 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:16.296586 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:16.297537 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:16.298933 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:16.299206 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:16.299338 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:16.302460 (+ 3122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:16.302467 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:16.302474 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:16.302514 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:16.304388 (+ 1874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:16.305110 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:16.305114 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:16.305116 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:16.305210 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:16.305685 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:16.305691 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:16.305694 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:16.306006 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:16.306118 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:16.306927 (+ 809us) server_negotiation.cc:299] Negotiation successful
0806 11:36:16.307106 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260806 11:36:16.308171 22838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:16.293076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:40925 (local address 127.21.14.193:35379)
0806 11:36:16.293634 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:16.293682 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:16.294670 (+ 988us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:16.295385 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:16.295395 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:16.295830 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:16.296394 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:16.296411 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:16.297737 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:16.297740 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:16.298755 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:16.298768 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:16.299412 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:16.299416 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:16.299617 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:16.300350 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:16.300372 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:16.302182 (+ 1810us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:16.304556 (+ 2374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:16.304563 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:16.304579 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:16.304955 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:16.305340 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:16.305344 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:16.305346 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:16.305542 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:16.306150 (+ 608us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:16.306157 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:16.306559 (+ 402us) client_negotiation.cc:769] Sending connection context
0806 11:36:16.306774 (+ 215us) client_negotiation.cc:240] Negotiation successful
0806 11:36:16.307179 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":145,"threads_started":1}
I20260806 11:36:16.309748 22950 heartbeater.cc:347] Connected to a master server at 127.21.14.254:40925
I20260806 11:36:16.309989 22950 heartbeater.cc:464] Registering TS with master...
I20260806 11:36:16.310707 22950 heartbeater.cc:511] Master 127.21.14.254:40925 requested a full tablet report, sending...
I20260806 11:36:16.312819 22736 ts_manager.cc:194] Registered new tserver with Master: 5b2c000f19594cfa8696548a481014b7 (127.21.14.193:40535)
I20260806 11:36:16.312902 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 22813
I20260806 11:36:16.313061 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/ts-0/wal/instance
I20260806 11:36:16.313328 21563 external_mini_cluster.cc:1428] Setting key 950c6fa7fbe8f0ac1ad409c102c37bd1
I20260806 11:36:16.314451 22736 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.14.193@KRBTEST.COM'} at 127.21.14.193:35379
I20260806 11:36:16.321615 22957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:16.312624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:40925 (local address 127.21.14.193:59205)
0806 11:36:16.313166 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:16.313182 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:16.313324 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:16.313684 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:16.313687 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:16.313948 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:16.314079 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:16.314086 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:16.315019 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:16.315023 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:16.315757 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:16.315767 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:16.316239 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:16.316242 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:16.316338 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:16.317070 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:16.317088 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:16.317628 (+ 540us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:16.319873 (+ 2245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:16.319877 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:16.319879 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:16.320122 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:16.320443 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:16.320447 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:16.320449 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:16.320548 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:16.320996 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:16.321002 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:16.321109 (+ 107us) client_negotiation.cc:769] Sending connection context
0806 11:36:16.321223 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 11:36:16.321408 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"mutex_wait_us":3,"thread_start_us":120,"threads_started":1}
I20260806 11:36:16.321605 22789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:16.312766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:59205 (local address 127.21.14.254:40925)
0806 11:36:16.312981 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:16.312986 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:16.313251 (+ 265us) server_negotiation.cc:407] Connection header received
0806 11:36:16.313434 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:16.313437 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:16.313482 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:16.313532 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:16.314226 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:16.314839 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:16.315885 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:16.316115 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:16.316232 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:16.317834 (+ 1602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:16.317838 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:16.317840 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:16.317878 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:16.319708 (+ 1830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:16.320242 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:16.320246 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:16.320248 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:16.320340 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:16.320649 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:16.320652 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:16.320654 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:16.320871 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:16.320993 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:16.321284 (+ 291us) server_negotiation.cc:299] Negotiation successful
0806 11:36:16.321415 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
WARNING: no policy specified for kudu/127.21.14.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab.
Entry for principal kudu/127.21.14.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab.
I20260806 11:36:16.351174 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.194:0
--local_ip_for_outbound_sockets=127.21.14.194
--webserver_interface=127.21.14.194
--webserver_port=0
--tserver_master_addrs=127.21.14.254:40925
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab
--principal=kudu/127.21.14.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:37439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:36:16.475904 22960 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:36:16.476243 22960 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:36:16.476290 22960 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:36:16.480590 22960 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:36:16.480666 22960 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:36:16.480708 22960 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 11:36:16.480743 22960 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.194
I20260806 11:36:16.486567 22960 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:37439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab
--principal=kudu/127.21.14.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.14.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.14.254:40925
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22960
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.194
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:36:16.487818 22960 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:36:16.488860 22960 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:36:16.499225 22966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:16.499238 22967 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:16.499244 22970 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:16.499512 22960 server_base.cc:1034] running on GCE node
I20260806 11:36:16.500083 22960 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:36:16.500808 22960 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:36:16.502067 22960 hybrid_clock.cc:648] HybridClock initialized: now 1786016176502039 us; error 36 us; skew 500 ppm
Aug 06 11:36:16 dist-test-slave-7c35 krb5kdc[22690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016176, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:36:16.505425 22960 init.cc:377] Logged in from keytab as kudu/127.21.14.194@KRBTEST.COM (short username kudu)
I20260806 11:36:16.506440 22960 webserver.cc:460] Webserver started at http://127.21.14.194:38767/ using document root <none> and password file <none>
I20260806 11:36:16.507138 22960 fs_manager.cc:359] Metadata directory not provided
I20260806 11:36:16.507236 22960 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:36:16.507475 22960 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:36:16.509701 22960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/ts-1/data/instance:
uuid: "f32715fb8fff45afb9e03ef09c7eb305"
format_stamp: "Formatted at 2026-08-06 11:36:16 on dist-test-slave-7c35"
server_key: "9dbdd8bf6d0c71ee2abdf34b80d845a9"
server_key_iv: "969f7da35651786408db28c5da88f5cf"
server_key_version: "encryptionkey@0"
I20260806 11:36:16.510411 22960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/ts-1/wal/instance:
uuid: "f32715fb8fff45afb9e03ef09c7eb305"
format_stamp: "Formatted at 2026-08-06 11:36:16 on dist-test-slave-7c35"
server_key: "9dbdd8bf6d0c71ee2abdf34b80d845a9"
server_key_iv: "969f7da35651786408db28c5da88f5cf"
server_key_version: "encryptionkey@0"
I20260806 11:36:16.514586 22960 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260806 11:36:16.517378 22977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:16.518831 22960 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 11:36:16.519078 22960 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/ts-1/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/ts-1/wal
uuid: "f32715fb8fff45afb9e03ef09c7eb305"
format_stamp: "Formatted at 2026-08-06 11:36:16 on dist-test-slave-7c35"
server_key: "9dbdd8bf6d0c71ee2abdf34b80d845a9"
server_key_iv: "969f7da35651786408db28c5da88f5cf"
server_key_version: "encryptionkey@0"
I20260806 11:36:16.519204 22960 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:36:16.543505 22960 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:36:16.547139 22960 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:36:16.547355 22960 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:36:16.548946 22960 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:36:16.549036 22960 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:16.549110 22960 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:36:16.549157 22960 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 11:36:16 dist-test-slave-7c35 krb5kdc[22690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016176, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.194@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:36:16.561512 22960 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.194:40175
I20260806 11:36:16.562333 23097 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.194:40175 every 8 connection(s)
I20260806 11:36:16.562556 22960 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
I20260806 11:36:16.564576 22789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:16.549990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:60337 (local address 127.21.14.254:40925)
0806 11:36:16.550175 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:16.550181 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:16.551170 (+ 989us) server_negotiation.cc:407] Connection header received
0806 11:36:16.552313 (+ 1143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:16.552317 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:16.552390 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:16.552487 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:16.553954 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:16.554524 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:16.556097 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:16.556392 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:16.556546 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:16.560285 (+ 3739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:16.560293 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:16.560299 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:16.560333 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:16.562162 (+ 1829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:16.562808 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:16.562812 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:16.562814 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:16.562957 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:16.563375 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:16.563379 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:16.563380 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:16.563645 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:16.563742 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:16.564293 (+ 551us) server_negotiation.cc:299] Negotiation successful
0806 11:36:16.564413 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 11:36:16.565441 22986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:16.550438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:40925 (local address 127.21.14.194:60337)
0806 11:36:16.550987 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:16.551046 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:16.552072 (+ 1026us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:16.552754 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:16.552761 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:16.553252 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:16.553782 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:16.553798 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:16.554673 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:16.554677 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:16.555904 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:16.555915 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:16.556664 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:16.556672 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:16.556995 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:16.557800 (+ 805us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:16.557822 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:16.560059 (+ 2237us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:16.562317 (+ 2258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:16.562325 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:16.562344 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:16.562698 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:16.563063 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:16.563068 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:16.563070 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:16.563242 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:16.563789 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:16.563795 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:16.564060 (+ 265us) client_negotiation.cc:769] Sending connection context
0806 11:36:16.564231 (+ 171us) client_negotiation.cc:240] Negotiation successful
0806 11:36:16.564541 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":148,"threads_started":1}
I20260806 11:36:16.566705 23098 heartbeater.cc:347] Connected to a master server at 127.21.14.254:40925
I20260806 11:36:16.566900 23098 heartbeater.cc:464] Registering TS with master...
I20260806 11:36:16.567605 23098 heartbeater.cc:511] Master 127.21.14.254:40925 requested a full tablet report, sending...
I20260806 11:36:16.568919 22739 ts_manager.cc:194] Registered new tserver with Master: f32715fb8fff45afb9e03ef09c7eb305 (127.21.14.194:40175)
I20260806 11:36:16.569720 22739 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.14.194@KRBTEST.COM'} at 127.21.14.194:60337
I20260806 11:36:16.569943 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 22960
I20260806 11:36:16.570055 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/ts-1/wal/instance
I20260806 11:36:16.570411 21563 external_mini_cluster.cc:1428] Setting key b797f29547265bc40097d961aaf26f83
I20260806 11:36:16.578197 22789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:16.569427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:33099 (local address 127.21.14.254:40925)
0806 11:36:16.569577 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:16.569581 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:16.569848 (+ 267us) server_negotiation.cc:407] Connection header received
0806 11:36:16.569939 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:16.569942 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:16.570001 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:16.570104 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:16.570832 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:16.571504 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:16.572716 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:16.572934 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:16.573021 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:16.574471 (+ 1450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:16.574477 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:16.574482 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:16.574519 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:16.576205 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:16.576773 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:16.576776 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:16.576777 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:16.576852 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:16.577190 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:16.577193 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:16.577194 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:16.577441 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:16.577593 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:16.577886 (+ 293us) server_negotiation.cc:299] Negotiation successful
0806 11:36:16.578031 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 11:36:16.578212 23105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:16.569360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:40925 (local address 127.21.14.194:33099)
0806 11:36:16.569703 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:16.569720 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:16.569837 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:16.570281 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:16.570284 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:16.570544 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:16.570679 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:16.570690 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:16.571717 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:16.571720 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:16.572568 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:16.572583 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:16.573053 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:16.573059 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:16.573192 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:16.573763 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:16.573784 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:16.574314 (+ 530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:16.576334 (+ 2020us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:16.576340 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:16.576345 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:16.576672 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:16.576941 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:16.576947 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:16.576952 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:16.577061 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:16.577582 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:16.577586 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:16.577691 (+ 105us) client_negotiation.cc:769] Sending connection context
0806 11:36:16.577817 (+ 126us) client_negotiation.cc:240] Negotiation successful
0806 11:36:16.577999 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":125,"threads_started":1}
WARNING: no policy specified for kudu/127.21.14.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab.
Entry for principal kudu/127.21.14.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab.
I20260806 11:36:16.611775 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.195:0
--local_ip_for_outbound_sockets=127.21.14.195
--webserver_interface=127.21.14.195
--webserver_port=0
--tserver_master_addrs=127.21.14.254:40925
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab
--principal=kudu/127.21.14.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:37439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:36:16.735353 23108 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:36:16.735679 23108 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:36:16.735728 23108 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:36:16.740108 23108 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:36:16.740188 23108 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:36:16.740232 23108 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 11:36:16.740268 23108 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.195
I20260806 11:36:16.745954 23108 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:37439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab
--principal=kudu/127.21.14.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.14.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.14.254:40925
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23108
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.195
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:36:16.747260 23108 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:36:16.748310 23108 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:36:16.759035 23114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:16.759035 23115 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:16.759111 23108 server_base.cc:1034] running on GCE node
W20260806 11:36:16.759416 23117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:16.759910 23108 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:36:16.760653 23108 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:36:16.761861 23108 hybrid_clock.cc:648] HybridClock initialized: now 1786016176761822 us; error 51 us; skew 500 ppm
Aug 06 11:36:16 dist-test-slave-7c35 krb5kdc[22690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016176, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:36:16.765188 23108 init.cc:377] Logged in from keytab as kudu/127.21.14.195@KRBTEST.COM (short username kudu)
I20260806 11:36:16.766384 23108 webserver.cc:460] Webserver started at http://127.21.14.195:34041/ using document root <none> and password file <none>
I20260806 11:36:16.767110 23108 fs_manager.cc:359] Metadata directory not provided
I20260806 11:36:16.767207 23108 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:36:16.767488 23108 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:36:16.769695 23108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/ts-2/data/instance:
uuid: "ce59d498c5d045519b0d4f5306c4e46d"
format_stamp: "Formatted at 2026-08-06 11:36:16 on dist-test-slave-7c35"
server_key: "eba99b4af22bed640cb476d203894562"
server_key_iv: "cc0c44fd05537cd14f5c0c2f9978849d"
server_key_version: "encryptionkey@0"
I20260806 11:36:16.770308 23108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/ts-2/wal/instance:
uuid: "ce59d498c5d045519b0d4f5306c4e46d"
format_stamp: "Formatted at 2026-08-06 11:36:16 on dist-test-slave-7c35"
server_key: "eba99b4af22bed640cb476d203894562"
server_key_iv: "cc0c44fd05537cd14f5c0c2f9978849d"
server_key_version: "encryptionkey@0"
I20260806 11:36:16.774659 23108 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260806 11:36:16.777552 23124 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:16.779104 23108 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 11:36:16.779258 23108 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/ts-2/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/ts-2/wal
uuid: "ce59d498c5d045519b0d4f5306c4e46d"
format_stamp: "Formatted at 2026-08-06 11:36:16 on dist-test-slave-7c35"
server_key: "eba99b4af22bed640cb476d203894562"
server_key_iv: "cc0c44fd05537cd14f5c0c2f9978849d"
server_key_version: "encryptionkey@0"
I20260806 11:36:16.779386 23108 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:36:16.793826 23108 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:36:16.797298 23108 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:36:16.797534 23108 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:36:16.799185 23108 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:36:16.799257 23108 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:16.799327 23108 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:36:16.799366 23108 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 11:36:16 dist-test-slave-7c35 krb5kdc[22690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016176, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.195@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:36:16.812325 23108 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.195:36341
I20260806 11:36:16.812354 23241 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.195:36341 every 8 connection(s)
I20260806 11:36:16.813364 23108 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
I20260806 11:36:16.815601 22789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:16.800196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:48487 (local address 127.21.14.254:40925)
0806 11:36:16.800326 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:16.800330 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:16.801588 (+ 1258us) server_negotiation.cc:407] Connection header received
0806 11:36:16.802812 (+ 1224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:16.802816 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:16.802934 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:16.803016 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:16.804671 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:16.805612 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:16.807298 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:16.807818 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:16.808073 (+ 255us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:16.811402 (+ 3329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:16.811406 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:16.811409 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:16.811444 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:16.813190 (+ 1746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:16.813818 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:16.813822 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:16.813829 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:16.813908 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:16.814317 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:16.814320 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:16.814322 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:16.814601 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:16.814700 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:16.815295 (+ 595us) server_negotiation.cc:299] Negotiation successful
0806 11:36:16.815431 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 11:36:16.816486 23133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:16.800818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:40925 (local address 127.21.14.195:48487)
0806 11:36:16.801406 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:16.801457 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:16.802551 (+ 1094us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:16.803287 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:16.803296 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:16.803857 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:16.804470 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:16.804500 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:16.805780 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:16.805784 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:16.807098 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:16.807111 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:16.808071 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:16.808078 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:16.808328 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:16.809177 (+ 849us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:16.809232 (+ 55us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:16.811153 (+ 1921us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:16.813328 (+ 2175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:16.813334 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:16.813351 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:16.813699 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:16.814025 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:16.814028 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:16.814030 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:16.814206 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:16.814706 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:16.814715 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:16.815039 (+ 324us) client_negotiation.cc:769] Sending connection context
0806 11:36:16.815235 (+ 196us) client_negotiation.cc:240] Negotiation successful
0806 11:36:16.815596 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":165,"threads_started":1}
I20260806 11:36:16.817965 23242 heartbeater.cc:347] Connected to a master server at 127.21.14.254:40925
I20260806 11:36:16.818153 23242 heartbeater.cc:464] Registering TS with master...
I20260806 11:36:16.818748 23242 heartbeater.cc:511] Master 127.21.14.254:40925 requested a full tablet report, sending...
I20260806 11:36:16.820014 22736 ts_manager.cc:194] Registered new tserver with Master: ce59d498c5d045519b0d4f5306c4e46d (127.21.14.195:36341)
I20260806 11:36:16.820436 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 23108
I20260806 11:36:16.820564 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/ts-2/wal/instance
I20260806 11:36:16.820899 21563 external_mini_cluster.cc:1428] Setting key c183b160d801c74e269e5cf829a36f48
I20260806 11:36:16.820870 22736 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.14.195@KRBTEST.COM'} at 127.21.14.195:48487
I20260806 11:36:16.823508 21563 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 11:36:16.829104 23248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:16.820387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:40925 (local address 127.21.14.195:33083)
0806 11:36:16.820899 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:16.820915 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:16.821058 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:16.821363 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:16.821367 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:16.821657 (+ 290us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:16.821803 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:16.821810 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:16.822660 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:16.822663 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:16.823498 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:16.823515 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:16.824070 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:16.824073 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:16.824149 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:16.824838 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:16.824858 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:16.825284 (+ 426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:16.827453 (+ 2169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:16.827456 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:16.827457 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:16.827674 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:16.828032 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:16.828034 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:16.828035 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:16.828111 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:16.828636 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:16.828638 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:16.828724 (+ 86us) client_negotiation.cc:769] Sending connection context
0806 11:36:16.828804 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 11:36:16.828930 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":421,"mutex_wait_us":3,"thread_start_us":130,"threads_started":1}
I20260806 11:36:16.829344 22789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:16.820701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:33083 (local address 127.21.14.254:40925)
0806 11:36:16.820845 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:16.820850 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:16.820983 (+ 133us) server_negotiation.cc:407] Connection header received
0806 11:36:16.821176 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:16.821179 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:16.821234 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:16.821333 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:16.821922 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:16.822524 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:16.823626 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:16.823935 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:16.824069 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:16.825453 (+ 1384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:16.825459 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:16.825466 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:16.825502 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:16.827262 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:16.827787 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:16.827794 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:16.827799 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:16.827918 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:16.828202 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:16.828208 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:16.828213 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:16.828528 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:16.828660 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:16.828912 (+ 252us) server_negotiation.cc:299] Negotiation successful
0806 11:36:16.829087 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
Aug 06 11:36:16 dist-test-slave-7c35 krb5kdc[22690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016176, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 11:36:16 dist-test-slave-7c35 krb5kdc[22690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016176, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:36:16.853542 22789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:16.845164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50350 (local address 127.21.14.254:40925)
0806 11:36:16.845308 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:16.845311 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:16.845323 (+ 12us) server_negotiation.cc:407] Connection header received
0806 11:36:16.845367 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:16.845369 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:16.845410 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:16.845482 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:16.846248 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:16.846920 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:16.847779 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:16.848007 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:16.848082 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:16.850578 (+ 2496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:16.850581 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:16.850583 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:16.850612 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:16.852011 (+ 1399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:16.852450 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:16.852453 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:16.852454 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:16.852532 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:16.852772 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:16.852774 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:16.852775 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:16.852973 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:16.853076 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:16.853306 (+ 230us) server_negotiation.cc:299] Negotiation successful
0806 11:36:16.853410 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 11:36:16.856747 22736 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50350:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 11:36:16.859982 22736 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 11:36:16.876026 23265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:16.869971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.195:36341 (local address 127.0.0.1:33084)
0806 11:36:16.871022 (+ 1051us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:16.871037 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:16.871116 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:16.871898 (+ 782us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:16.871901 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:16.871919 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:36:16.872022 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:16.872028 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:16.873123 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:16.873130 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:16.874336 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:16.874346 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:16.875404 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:16.875408 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:16.875535 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:16.875584 (+ 49us) client_negotiation.cc:769] Sending connection context
0806 11:36:16.875699 (+ 115us) client_negotiation.cc:240] Negotiation successful
0806 11:36:16.875829 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":986,"mutex_wait_us":68,"thread_start_us":122,"threads_started":1}
I20260806 11:36:16.876055 23268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:16.870680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33084 (local address 127.21.14.195:36341)
0806 11:36:16.871493 (+ 813us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:16.871510 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:16.871529 (+ 19us) server_negotiation.cc:407] Connection header received
0806 11:36:16.871623 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:16.871630 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:16.871774 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:16.871925 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:36:16.872151 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:16.873014 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:16.874472 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:16.875269 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:16.875385 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:16.875566 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:16.875752 (+ 186us) server_negotiation.cc:299] Negotiation successful
0806 11:36:16.875862 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":699,"thread_start_us":107,"threads_started":1}
I20260806 11:36:16.876415 23267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:16.870225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46564 (local address 127.21.14.193:40535)
0806 11:36:16.870500 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:16.870507 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:16.870773 (+ 266us) server_negotiation.cc:407] Connection header received
0806 11:36:16.871140 (+ 367us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:16.871147 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:16.871341 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:16.871475 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:36:16.871935 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:16.873034 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:16.874932 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:16.875560 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:16.875677 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:16.875822 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:16.876159 (+ 337us) server_negotiation.cc:299] Negotiation successful
0806 11:36:16.876269 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":91,"threads_started":1}
I20260806 11:36:16.876701 23262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:16.869337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.194:40175 (local address 127.0.0.1:42554)
0806 11:36:16.869851 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:16.869901 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:16.870050 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:16.871137 (+ 1087us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:16.871143 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:16.871169 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:36:16.871331 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:16.871341 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:16.872946 (+ 1605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:16.872949 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:16.874640 (+ 1691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:16.874651 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:16.875984 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:16.875987 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:16.876263 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:16.876287 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 11:36:16.876358 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 11:36:16.876430 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"mutex_wait_us":166,"thread_start_us":85,"threads_started":1}
I20260806 11:36:16.876879 23266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:16.869535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.193:40535 (local address 127.0.0.1:46564)
0806 11:36:16.870667 (+ 1132us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:16.870683 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:16.870852 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:16.871673 (+ 821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:16.871676 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:16.871694 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:36:16.871803 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:16.871809 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:16.873170 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:16.873173 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:16.874640 (+ 1467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:16.874651 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:16.875886 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:16.875890 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:16.875992 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:16.876014 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 11:36:16.876631 (+ 617us) client_negotiation.cc:240] Negotiation successful
0806 11:36:16.876719 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1066,"mutex_wait_us":122,"thread_start_us":98,"threads_started":1}
I20260806 11:36:16.877187 23263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:16.869478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42554 (local address 127.21.14.194:40175)
0806 11:36:16.870561 (+ 1083us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:16.870583 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:16.870611 (+ 28us) server_negotiation.cc:407] Connection header received
0806 11:36:16.870693 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:16.870699 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:16.870958 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:16.871137 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:36:16.871465 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:16.872794 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:16.874775 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:16.875639 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:16.875764 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:16.875917 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:16.876952 (+ 1035us) server_negotiation.cc:299] Negotiation successful
0806 11:36:16.877054 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":975,"thread_start_us":150,"threads_started":1}
I20260806 11:36:16.879763 22881 tablet_service.cc:1467] Processing CreateTablet for tablet c2e92516c2294258a2e3d8cb31249655 (DEFAULT_TABLE table=test-table [id=dfb301a9dfd94b63980c94ba92f17522]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:36:16.880030 23029 tablet_service.cc:1467] Processing CreateTablet for tablet c2e92516c2294258a2e3d8cb31249655 (DEFAULT_TABLE table=test-table [id=dfb301a9dfd94b63980c94ba92f17522]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:36:16.880029 23176 tablet_service.cc:1467] Processing CreateTablet for tablet c2e92516c2294258a2e3d8cb31249655 (DEFAULT_TABLE table=test-table [id=dfb301a9dfd94b63980c94ba92f17522]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:36:16.880919 22881 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2e92516c2294258a2e3d8cb31249655. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:16.881218 23176 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2e92516c2294258a2e3d8cb31249655. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:16.881212 23029 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2e92516c2294258a2e3d8cb31249655. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:16.888269 23269 tablet_bootstrap.cc:492] T c2e92516c2294258a2e3d8cb31249655 P 5b2c000f19594cfa8696548a481014b7: Bootstrap starting.
I20260806 11:36:16.890529 23270 tablet_bootstrap.cc:492] T c2e92516c2294258a2e3d8cb31249655 P ce59d498c5d045519b0d4f5306c4e46d: Bootstrap starting.
I20260806 11:36:16.890923 23269 tablet_bootstrap.cc:654] T c2e92516c2294258a2e3d8cb31249655 P 5b2c000f19594cfa8696548a481014b7: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:16.891700 23271 tablet_bootstrap.cc:492] T c2e92516c2294258a2e3d8cb31249655 P f32715fb8fff45afb9e03ef09c7eb305: Bootstrap starting.
I20260806 11:36:16.892023 23269 log.cc:824] T c2e92516c2294258a2e3d8cb31249655 P 5b2c000f19594cfa8696548a481014b7: Log is configured to *not* fsync() on all Append() calls
I20260806 11:36:16.893054 23270 tablet_bootstrap.cc:654] T c2e92516c2294258a2e3d8cb31249655 P ce59d498c5d045519b0d4f5306c4e46d: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:16.893999 23270 log.cc:824] T c2e92516c2294258a2e3d8cb31249655 P ce59d498c5d045519b0d4f5306c4e46d: Log is configured to *not* fsync() on all Append() calls
I20260806 11:36:16.894241 23271 tablet_bootstrap.cc:654] T c2e92516c2294258a2e3d8cb31249655 P f32715fb8fff45afb9e03ef09c7eb305: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:16.894481 23269 tablet_bootstrap.cc:492] T c2e92516c2294258a2e3d8cb31249655 P 5b2c000f19594cfa8696548a481014b7: No bootstrap required, opened a new log
I20260806 11:36:16.894714 23269 ts_tablet_manager.cc:1397] T c2e92516c2294258a2e3d8cb31249655 P 5b2c000f19594cfa8696548a481014b7: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260806 11:36:16.895298 23271 log.cc:824] T c2e92516c2294258a2e3d8cb31249655 P f32715fb8fff45afb9e03ef09c7eb305: Log is configured to *not* fsync() on all Append() calls
I20260806 11:36:16.896705 23270 tablet_bootstrap.cc:492] T c2e92516c2294258a2e3d8cb31249655 P ce59d498c5d045519b0d4f5306c4e46d: No bootstrap required, opened a new log
I20260806 11:36:16.896924 23270 ts_tablet_manager.cc:1397] T c2e92516c2294258a2e3d8cb31249655 P ce59d498c5d045519b0d4f5306c4e46d: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260806 11:36:16.897356 23271 tablet_bootstrap.cc:492] T c2e92516c2294258a2e3d8cb31249655 P f32715fb8fff45afb9e03ef09c7eb305: No bootstrap required, opened a new log
I20260806 11:36:16.897578 23271 ts_tablet_manager.cc:1397] T c2e92516c2294258a2e3d8cb31249655 P f32715fb8fff45afb9e03ef09c7eb305: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 11:36:16.899910 23269 raft_consensus.cc:348] T c2e92516c2294258a2e3d8cb31249655 P 5b2c000f19594cfa8696548a481014b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f32715fb8fff45afb9e03ef09c7eb305" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 40175 } } peers { permanent_uuid: "5b2c000f19594cfa8696548a481014b7" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40535 } } peers { permanent_uuid: "ce59d498c5d045519b0d4f5306c4e46d" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 36341 } }
I20260806 11:36:16.900274 23269 raft_consensus.cc:374] T c2e92516c2294258a2e3d8cb31249655 P 5b2c000f19594cfa8696548a481014b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:16.900331 23269 raft_consensus.cc:729] T c2e92516c2294258a2e3d8cb31249655 P 5b2c000f19594cfa8696548a481014b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b2c000f19594cfa8696548a481014b7, State: Initialized, Role: FOLLOWER
I20260806 11:36:16.900892 23269 consensus_queue.cc:260] T c2e92516c2294258a2e3d8cb31249655 P 5b2c000f19594cfa8696548a481014b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f32715fb8fff45afb9e03ef09c7eb305" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 40175 } } peers { permanent_uuid: "5b2c000f19594cfa8696548a481014b7" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40535 } } peers { permanent_uuid: "ce59d498c5d045519b0d4f5306c4e46d" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 36341 } }
I20260806 11:36:16.901077 23270 raft_consensus.cc:348] T c2e92516c2294258a2e3d8cb31249655 P ce59d498c5d045519b0d4f5306c4e46d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f32715fb8fff45afb9e03ef09c7eb305" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 40175 } } peers { permanent_uuid: "5b2c000f19594cfa8696548a481014b7" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40535 } } peers { permanent_uuid: "ce59d498c5d045519b0d4f5306c4e46d" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 36341 } }
I20260806 11:36:16.901342 23270 raft_consensus.cc:374] T c2e92516c2294258a2e3d8cb31249655 P ce59d498c5d045519b0d4f5306c4e46d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:16.901400 23270 raft_consensus.cc:729] T c2e92516c2294258a2e3d8cb31249655 P ce59d498c5d045519b0d4f5306c4e46d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce59d498c5d045519b0d4f5306c4e46d, State: Initialized, Role: FOLLOWER
I20260806 11:36:16.901218 23271 raft_consensus.cc:348] T c2e92516c2294258a2e3d8cb31249655 P f32715fb8fff45afb9e03ef09c7eb305 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f32715fb8fff45afb9e03ef09c7eb305" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 40175 } } peers { permanent_uuid: "5b2c000f19594cfa8696548a481014b7" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40535 } } peers { permanent_uuid: "ce59d498c5d045519b0d4f5306c4e46d" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 36341 } }
I20260806 11:36:16.901556 23271 raft_consensus.cc:374] T c2e92516c2294258a2e3d8cb31249655 P f32715fb8fff45afb9e03ef09c7eb305 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:16.901643 23271 raft_consensus.cc:729] T c2e92516c2294258a2e3d8cb31249655 P f32715fb8fff45afb9e03ef09c7eb305 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f32715fb8fff45afb9e03ef09c7eb305, State: Initialized, Role: FOLLOWER
I20260806 11:36:16.901898 23270 consensus_queue.cc:260] T c2e92516c2294258a2e3d8cb31249655 P ce59d498c5d045519b0d4f5306c4e46d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f32715fb8fff45afb9e03ef09c7eb305" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 40175 } } peers { permanent_uuid: "5b2c000f19594cfa8696548a481014b7" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40535 } } peers { permanent_uuid: "ce59d498c5d045519b0d4f5306c4e46d" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 36341 } }
I20260806 11:36:16.902105 23269 ts_tablet_manager.cc:1428] T c2e92516c2294258a2e3d8cb31249655 P 5b2c000f19594cfa8696548a481014b7: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260806 11:36:16.902117 22950 heartbeater.cc:503] Master 127.21.14.254:40925 was elected leader, sending a full tablet report...
I20260806 11:36:16.902266 23271 consensus_queue.cc:260] T c2e92516c2294258a2e3d8cb31249655 P f32715fb8fff45afb9e03ef09c7eb305 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f32715fb8fff45afb9e03ef09c7eb305" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 40175 } } peers { permanent_uuid: "5b2c000f19594cfa8696548a481014b7" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40535 } } peers { permanent_uuid: "ce59d498c5d045519b0d4f5306c4e46d" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 36341 } }
I20260806 11:36:16.902714 23242 heartbeater.cc:503] Master 127.21.14.254:40925 was elected leader, sending a full tablet report...
I20260806 11:36:16.903061 23098 heartbeater.cc:503] Master 127.21.14.254:40925 was elected leader, sending a full tablet report...
I20260806 11:36:16.903558 23271 ts_tablet_manager.cc:1428] T c2e92516c2294258a2e3d8cb31249655 P f32715fb8fff45afb9e03ef09c7eb305: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 11:36:16.904371 23270 ts_tablet_manager.cc:1428] T c2e92516c2294258a2e3d8cb31249655 P ce59d498c5d045519b0d4f5306c4e46d: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260806 11:36:17.021761 23277 raft_consensus.cc:482] T c2e92516c2294258a2e3d8cb31249655 P f32715fb8fff45afb9e03ef09c7eb305 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 11:36:17.022013 23277 raft_consensus.cc:504] T c2e92516c2294258a2e3d8cb31249655 P f32715fb8fff45afb9e03ef09c7eb305 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f32715fb8fff45afb9e03ef09c7eb305" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 40175 } } peers { permanent_uuid: "5b2c000f19594cfa8696548a481014b7" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40535 } } peers { permanent_uuid: "ce59d498c5d045519b0d4f5306c4e46d" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 36341 } }
I20260806 11:36:17.023576 23277 leader_election.cc:288] T c2e92516c2294258a2e3d8cb31249655 P f32715fb8fff45afb9e03ef09c7eb305 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b2c000f19594cfa8696548a481014b7 (127.21.14.193:40535), ce59d498c5d045519b0d4f5306c4e46d (127.21.14.195:36341)
I20260806 11:36:17.028110 22986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:17.023856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.193:40535 (local address 127.21.14.194:51123)
0806 11:36:17.023990 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:17.024005 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:17.024127 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:17.024488 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:17.024491 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:17.024511 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:36:17.024642 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:17.024648 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:17.025412 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:17.025415 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:17.026551 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:17.026562 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:17.027728 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:17.027731 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:17.027822 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:17.027845 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 11:36:17.027917 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 11:36:17.027980 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260806 11:36:17.028183 23267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:17.023961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:51123 (local address 127.21.14.193:40535)
0806 11:36:17.024081 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:17.024085 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:17.024100 (+ 15us) server_negotiation.cc:407] Connection header received
0806 11:36:17.024296 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:17.024300 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:17.024363 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:17.024473 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:36:17.024806 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:17.025330 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:17.026698 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:17.027583 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:17.027720 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:17.027777 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:17.027966 (+ 189us) server_negotiation.cc:299] Negotiation successful
0806 11:36:17.028047 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 11:36:17.028998 22902 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2e92516c2294258a2e3d8cb31249655" candidate_uuid: "f32715fb8fff45afb9e03ef09c7eb305" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b2c000f19594cfa8696548a481014b7" is_pre_election: true
I20260806 11:36:17.029410 22902 raft_consensus.cc:2454] T c2e92516c2294258a2e3d8cb31249655 P 5b2c000f19594cfa8696548a481014b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f32715fb8fff45afb9e03ef09c7eb305 in term 0.
I20260806 11:36:17.029637 23278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:17.023856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.195:36341 (local address 127.21.14.194:40347)
0806 11:36:17.024670 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:17.024686 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:17.024796 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:17.025053 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:17.025056 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:17.025088 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:36:17.025216 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:17.025221 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:17.026225 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:17.026228 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:17.028242 (+ 2014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:17.028256 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:17.029185 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:17.029189 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:17.029294 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:17.029324 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 11:36:17.029389 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 11:36:17.029448 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":746,"mutex_wait_us":367,"thread_start_us":138,"threads_started":1}
I20260806 11:36:17.029645 23268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:17.024110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:40347 (local address 127.21.14.195:36341)
0806 11:36:17.024322 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:17.024327 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:17.024777 (+ 450us) server_negotiation.cc:407] Connection header received
0806 11:36:17.024887 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:17.024890 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:17.024933 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:17.025041 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:36:17.025325 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:17.026096 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:17.028413 (+ 2317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:17.029002 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:17.029151 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:17.029206 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:17.029435 (+ 229us) server_negotiation.cc:299] Negotiation successful
0806 11:36:17.029527 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115}
I20260806 11:36:17.030028 22981 leader_election.cc:302] T c2e92516c2294258a2e3d8cb31249655 P f32715fb8fff45afb9e03ef09c7eb305 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b2c000f19594cfa8696548a481014b7, f32715fb8fff45afb9e03ef09c7eb305; no voters:
I20260806 11:36:17.030323 23277 raft_consensus.cc:2781] T c2e92516c2294258a2e3d8cb31249655 P f32715fb8fff45afb9e03ef09c7eb305 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 11:36:17.030419 23277 raft_consensus.cc:482] T c2e92516c2294258a2e3d8cb31249655 P f32715fb8fff45afb9e03ef09c7eb305 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 11:36:17.030345 23196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2e92516c2294258a2e3d8cb31249655" candidate_uuid: "f32715fb8fff45afb9e03ef09c7eb305" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce59d498c5d045519b0d4f5306c4e46d" is_pre_election: true
I20260806 11:36:17.030470 23277 raft_consensus.cc:3037] T c2e92516c2294258a2e3d8cb31249655 P f32715fb8fff45afb9e03ef09c7eb305 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:36:17.030656 23196 raft_consensus.cc:2454] T c2e92516c2294258a2e3d8cb31249655 P ce59d498c5d045519b0d4f5306c4e46d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f32715fb8fff45afb9e03ef09c7eb305 in term 0.
I20260806 11:36:17.031700 23277 raft_consensus.cc:504] T c2e92516c2294258a2e3d8cb31249655 P f32715fb8fff45afb9e03ef09c7eb305 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f32715fb8fff45afb9e03ef09c7eb305" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 40175 } } peers { permanent_uuid: "5b2c000f19594cfa8696548a481014b7" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40535 } } peers { permanent_uuid: "ce59d498c5d045519b0d4f5306c4e46d" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 36341 } }
I20260806 11:36:17.032128 23277 leader_election.cc:288] T c2e92516c2294258a2e3d8cb31249655 P f32715fb8fff45afb9e03ef09c7eb305 [CANDIDATE]: Term 1 election: Requested vote from peers 5b2c000f19594cfa8696548a481014b7 (127.21.14.193:40535), ce59d498c5d045519b0d4f5306c4e46d (127.21.14.195:36341)
I20260806 11:36:17.032521 23196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2e92516c2294258a2e3d8cb31249655" candidate_uuid: "f32715fb8fff45afb9e03ef09c7eb305" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce59d498c5d045519b0d4f5306c4e46d"
I20260806 11:36:17.032550 22902 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2e92516c2294258a2e3d8cb31249655" candidate_uuid: "f32715fb8fff45afb9e03ef09c7eb305" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b2c000f19594cfa8696548a481014b7"
I20260806 11:36:17.032670 23196 raft_consensus.cc:3037] T c2e92516c2294258a2e3d8cb31249655 P ce59d498c5d045519b0d4f5306c4e46d [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:36:17.032714 22902 raft_consensus.cc:3037] T c2e92516c2294258a2e3d8cb31249655 P 5b2c000f19594cfa8696548a481014b7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:36:17.033947 22902 raft_consensus.cc:2454] T c2e92516c2294258a2e3d8cb31249655 P 5b2c000f19594cfa8696548a481014b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f32715fb8fff45afb9e03ef09c7eb305 in term 1.
I20260806 11:36:17.034322 22981 leader_election.cc:302] T c2e92516c2294258a2e3d8cb31249655 P f32715fb8fff45afb9e03ef09c7eb305 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b2c000f19594cfa8696548a481014b7, f32715fb8fff45afb9e03ef09c7eb305; no voters:
I20260806 11:36:17.034363 23196 raft_consensus.cc:2454] T c2e92516c2294258a2e3d8cb31249655 P ce59d498c5d045519b0d4f5306c4e46d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f32715fb8fff45afb9e03ef09c7eb305 in term 1.
I20260806 11:36:17.034605 23277 raft_consensus.cc:2781] T c2e92516c2294258a2e3d8cb31249655 P f32715fb8fff45afb9e03ef09c7eb305 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:36:17.034982 23277 raft_consensus.cc:686] T c2e92516c2294258a2e3d8cb31249655 P f32715fb8fff45afb9e03ef09c7eb305 [term 1 LEADER]: Becoming Leader. State: Replica: f32715fb8fff45afb9e03ef09c7eb305, State: Running, Role: LEADER
I20260806 11:36:17.035384 23277 consensus_queue.cc:237] T c2e92516c2294258a2e3d8cb31249655 P f32715fb8fff45afb9e03ef09c7eb305 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f32715fb8fff45afb9e03ef09c7eb305" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 40175 } } peers { permanent_uuid: "5b2c000f19594cfa8696548a481014b7" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40535 } } peers { permanent_uuid: "ce59d498c5d045519b0d4f5306c4e46d" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 36341 } }
I20260806 11:36:17.038803 22734 catalog_manager.cc:5495] T c2e92516c2294258a2e3d8cb31249655 P f32715fb8fff45afb9e03ef09c7eb305 reported cstate change: term changed from 0 to 1, leader changed from <none> to f32715fb8fff45afb9e03ef09c7eb305 (127.21.14.194). New cstate: current_term: 1 leader_uuid: "f32715fb8fff45afb9e03ef09c7eb305" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f32715fb8fff45afb9e03ef09c7eb305" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 40175 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b2c000f19594cfa8696548a481014b7" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce59d498c5d045519b0d4f5306c4e46d" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 36341 } health_report { overall_health: UNKNOWN } } }
W20260806 11:36:17.057379 22951 tablet.cc:2369] T c2e92516c2294258a2e3d8cb31249655 P 5b2c000f19594cfa8696548a481014b7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 11:36:17.064227 23099 tablet.cc:2369] T c2e92516c2294258a2e3d8cb31249655 P f32715fb8fff45afb9e03ef09c7eb305: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 11:36:17.064904 23243 tablet.cc:2369] T c2e92516c2294258a2e3d8cb31249655 P ce59d498c5d045519b0d4f5306c4e46d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 11:36:17.081156 23263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:17.076583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42556 (local address 127.21.14.194:40175)
0806 11:36:17.076752 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:17.076759 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:17.076780 (+ 21us) server_negotiation.cc:407] Connection header received
0806 11:36:17.077062 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:17.077066 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:17.077130 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:17.077241 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 11:36:17.077557 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:17.078087 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:17.079129 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:17.079393 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:17.079636 (+ 243us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:17.080057 (+ 421us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 11:36:17.080643 (+ 586us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 11:36:17.080788 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:17.080910 (+ 122us) server_negotiation.cc:299] Negotiation successful
0806 11:36:17.080991 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 11:36:17.089277 23196 raft_consensus.cc:1261] T c2e92516c2294258a2e3d8cb31249655 P ce59d498c5d045519b0d4f5306c4e46d [term 1 FOLLOWER]: Refusing update from remote peer f32715fb8fff45afb9e03ef09c7eb305: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 11:36:17.089897 22902 raft_consensus.cc:1261] T c2e92516c2294258a2e3d8cb31249655 P 5b2c000f19594cfa8696548a481014b7 [term 1 FOLLOWER]: Refusing update from remote peer f32715fb8fff45afb9e03ef09c7eb305: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 11:36:17.090281 23277 consensus_queue.cc:1035] T c2e92516c2294258a2e3d8cb31249655 P f32715fb8fff45afb9e03ef09c7eb305 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce59d498c5d045519b0d4f5306c4e46d" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 36341 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 11:36:17.090651 23280 consensus_queue.cc:1035] T c2e92516c2294258a2e3d8cb31249655 P f32715fb8fff45afb9e03ef09c7eb305 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b2c000f19594cfa8696548a481014b7" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 11:36:17.118783 22734 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50350:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 11:36:17.119184 22734 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50350:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 11:36:17.123077 22734 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0569d29f335e4760ad00e3096cef03fc: Sending DeleteTablet for 3 replicas of tablet c2e92516c2294258a2e3d8cb31249655
I20260806 11:36:17.124498 22881 tablet_service.cc:1514] Processing DeleteTablet for tablet c2e92516c2294258a2e3d8cb31249655 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:36:17 UTC) from {username='kudu', principal='kudu/127.21.14.254@KRBTEST.COM'} at 127.0.0.1:46564
I20260806 11:36:17.124498 23029 tablet_service.cc:1514] Processing DeleteTablet for tablet c2e92516c2294258a2e3d8cb31249655 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:36:17 UTC) from {username='kudu', principal='kudu/127.21.14.254@KRBTEST.COM'} at 127.0.0.1:42554
I20260806 11:36:17.124575 23176 tablet_service.cc:1514] Processing DeleteTablet for tablet c2e92516c2294258a2e3d8cb31249655 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:36:17 UTC) from {username='kudu', principal='kudu/127.21.14.254@KRBTEST.COM'} at 127.0.0.1:33084
I20260806 11:36:17.125291 23294 tablet_replica.cc:330] T c2e92516c2294258a2e3d8cb31249655 P ce59d498c5d045519b0d4f5306c4e46d: stopping tablet replica
I20260806 11:36:17.125540 23296 tablet_replica.cc:330] T c2e92516c2294258a2e3d8cb31249655 P f32715fb8fff45afb9e03ef09c7eb305: stopping tablet replica
I20260806 11:36:17.125698 23294 raft_consensus.cc:2229] T c2e92516c2294258a2e3d8cb31249655 P ce59d498c5d045519b0d4f5306c4e46d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 11:36:17.125923 23296 raft_consensus.cc:2229] T c2e92516c2294258a2e3d8cb31249655 P f32715fb8fff45afb9e03ef09c7eb305 [term 1 LEADER]: Raft consensus shutting down.
I20260806 11:36:17.126096 23294 raft_consensus.cc:2258] T c2e92516c2294258a2e3d8cb31249655 P ce59d498c5d045519b0d4f5306c4e46d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 11:36:17.126475 23296 raft_consensus.cc:2258] T c2e92516c2294258a2e3d8cb31249655 P f32715fb8fff45afb9e03ef09c7eb305 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 11:36:17.126550 23295 tablet_replica.cc:330] T c2e92516c2294258a2e3d8cb31249655 P 5b2c000f19594cfa8696548a481014b7: stopping tablet replica
I20260806 11:36:17.126933 23295 raft_consensus.cc:2229] T c2e92516c2294258a2e3d8cb31249655 P 5b2c000f19594cfa8696548a481014b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 11:36:17.126922 22734 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50350:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 11:36:17.127271 23295 raft_consensus.cc:2258] T c2e92516c2294258a2e3d8cb31249655 P 5b2c000f19594cfa8696548a481014b7 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260806 11:36:17.127632 22734 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 11:36:17.128472 23294 ts_tablet_manager.cc:1905] T c2e92516c2294258a2e3d8cb31249655 P ce59d498c5d045519b0d4f5306c4e46d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 11:36:17.130239 23296 ts_tablet_manager.cc:1905] T c2e92516c2294258a2e3d8cb31249655 P f32715fb8fff45afb9e03ef09c7eb305: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 11:36:17.132262 23294 ts_tablet_manager.cc:1918] T c2e92516c2294258a2e3d8cb31249655 P ce59d498c5d045519b0d4f5306c4e46d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 11:36:17.132411 23294 log.cc:1196] T c2e92516c2294258a2e3d8cb31249655 P ce59d498c5d045519b0d4f5306c4e46d: Deleting WAL directory at /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/ts-2/wal/wals/c2e92516c2294258a2e3d8cb31249655
I20260806 11:36:17.132795 23294 ts_tablet_manager.cc:1939] T c2e92516c2294258a2e3d8cb31249655 P ce59d498c5d045519b0d4f5306c4e46d: Deleting consensus metadata
I20260806 11:36:17.134979 22881 tablet_service.cc:1467] Processing CreateTablet for tablet ddb6fd5c5f6e42cdbc4f615042249fbb (DEFAULT_TABLE table=test-table [id=e72288dc83e74235b2a4b709efceda40]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:36:17.135159 23176 tablet_service.cc:1467] Processing CreateTablet for tablet ddb6fd5c5f6e42cdbc4f615042249fbb (DEFAULT_TABLE table=test-table [id=e72288dc83e74235b2a4b709efceda40]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:36:17.135166 22720 catalog_manager.cc:4841] TS ce59d498c5d045519b0d4f5306c4e46d (127.21.14.195:36341): tablet c2e92516c2294258a2e3d8cb31249655 (table test-table [id=dfb301a9dfd94b63980c94ba92f17522]) successfully deleted
I20260806 11:36:17.135350 22881 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ddb6fd5c5f6e42cdbc4f615042249fbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:17.135442 23176 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ddb6fd5c5f6e42cdbc4f615042249fbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:17.135716 23029 tablet_service.cc:1467] Processing CreateTablet for tablet ddb6fd5c5f6e42cdbc4f615042249fbb (DEFAULT_TABLE table=test-table [id=e72288dc83e74235b2a4b709efceda40]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:36:17.136082 23029 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ddb6fd5c5f6e42cdbc4f615042249fbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:17.138343 23270 tablet_bootstrap.cc:492] T ddb6fd5c5f6e42cdbc4f615042249fbb P ce59d498c5d045519b0d4f5306c4e46d: Bootstrap starting.
I20260806 11:36:17.138800 23269 tablet_bootstrap.cc:492] T ddb6fd5c5f6e42cdbc4f615042249fbb P 5b2c000f19594cfa8696548a481014b7: Bootstrap starting.
I20260806 11:36:17.139535 23270 tablet_bootstrap.cc:654] T ddb6fd5c5f6e42cdbc4f615042249fbb P ce59d498c5d045519b0d4f5306c4e46d: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:17.139940 23295 ts_tablet_manager.cc:1905] T c2e92516c2294258a2e3d8cb31249655 P 5b2c000f19594cfa8696548a481014b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 11:36:17.139948 23269 tablet_bootstrap.cc:654] T ddb6fd5c5f6e42cdbc4f615042249fbb P 5b2c000f19594cfa8696548a481014b7: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:17.141161 23296 ts_tablet_manager.cc:1918] T c2e92516c2294258a2e3d8cb31249655 P f32715fb8fff45afb9e03ef09c7eb305: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 11:36:17.141292 23296 log.cc:1196] T c2e92516c2294258a2e3d8cb31249655 P f32715fb8fff45afb9e03ef09c7eb305: Deleting WAL directory at /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/ts-1/wal/wals/c2e92516c2294258a2e3d8cb31249655
I20260806 11:36:17.141659 23296 ts_tablet_manager.cc:1939] T c2e92516c2294258a2e3d8cb31249655 P f32715fb8fff45afb9e03ef09c7eb305: Deleting consensus metadata
I20260806 11:36:17.141935 23271 tablet_bootstrap.cc:492] T ddb6fd5c5f6e42cdbc4f615042249fbb P f32715fb8fff45afb9e03ef09c7eb305: Bootstrap starting.
I20260806 11:36:17.143208 23271 tablet_bootstrap.cc:654] T ddb6fd5c5f6e42cdbc4f615042249fbb P f32715fb8fff45afb9e03ef09c7eb305: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:17.143368 23270 tablet_bootstrap.cc:492] T ddb6fd5c5f6e42cdbc4f615042249fbb P ce59d498c5d045519b0d4f5306c4e46d: No bootstrap required, opened a new log
I20260806 11:36:17.143345 23295 ts_tablet_manager.cc:1918] T c2e92516c2294258a2e3d8cb31249655 P 5b2c000f19594cfa8696548a481014b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 11:36:17.143467 23295 log.cc:1196] T c2e92516c2294258a2e3d8cb31249655 P 5b2c000f19594cfa8696548a481014b7: Deleting WAL directory at /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/ts-0/wal/wals/c2e92516c2294258a2e3d8cb31249655
I20260806 11:36:17.143465 23270 ts_tablet_manager.cc:1397] T ddb6fd5c5f6e42cdbc4f615042249fbb P ce59d498c5d045519b0d4f5306c4e46d: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260806 11:36:17.143993 23270 raft_consensus.cc:348] T ddb6fd5c5f6e42cdbc4f615042249fbb P ce59d498c5d045519b0d4f5306c4e46d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2c000f19594cfa8696548a481014b7" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40535 } } peers { permanent_uuid: "f32715fb8fff45afb9e03ef09c7eb305" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 40175 } } peers { permanent_uuid: "ce59d498c5d045519b0d4f5306c4e46d" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 36341 } }
I20260806 11:36:17.144116 23270 raft_consensus.cc:374] T ddb6fd5c5f6e42cdbc4f615042249fbb P ce59d498c5d045519b0d4f5306c4e46d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:17.144147 23270 raft_consensus.cc:729] T ddb6fd5c5f6e42cdbc4f615042249fbb P ce59d498c5d045519b0d4f5306c4e46d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce59d498c5d045519b0d4f5306c4e46d, State: Initialized, Role: FOLLOWER
I20260806 11:36:17.144328 23270 consensus_queue.cc:260] T ddb6fd5c5f6e42cdbc4f615042249fbb P ce59d498c5d045519b0d4f5306c4e46d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2c000f19594cfa8696548a481014b7" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40535 } } peers { permanent_uuid: "f32715fb8fff45afb9e03ef09c7eb305" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 40175 } } peers { permanent_uuid: "ce59d498c5d045519b0d4f5306c4e46d" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 36341 } }
I20260806 11:36:17.144722 23295 ts_tablet_manager.cc:1939] T c2e92516c2294258a2e3d8cb31249655 P 5b2c000f19594cfa8696548a481014b7: Deleting consensus metadata
I20260806 11:36:17.144733 23270 ts_tablet_manager.cc:1428] T ddb6fd5c5f6e42cdbc4f615042249fbb P ce59d498c5d045519b0d4f5306c4e46d: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260806 11:36:17.145466 23269 tablet_bootstrap.cc:492] T ddb6fd5c5f6e42cdbc4f615042249fbb P 5b2c000f19594cfa8696548a481014b7: No bootstrap required, opened a new log
I20260806 11:36:17.145560 23269 ts_tablet_manager.cc:1397] T ddb6fd5c5f6e42cdbc4f615042249fbb P 5b2c000f19594cfa8696548a481014b7: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.002s
I20260806 11:36:17.146045 23269 raft_consensus.cc:348] T ddb6fd5c5f6e42cdbc4f615042249fbb P 5b2c000f19594cfa8696548a481014b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2c000f19594cfa8696548a481014b7" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40535 } } peers { permanent_uuid: "f32715fb8fff45afb9e03ef09c7eb305" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 40175 } } peers { permanent_uuid: "ce59d498c5d045519b0d4f5306c4e46d" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 36341 } }
I20260806 11:36:17.146163 23269 raft_consensus.cc:374] T ddb6fd5c5f6e42cdbc4f615042249fbb P 5b2c000f19594cfa8696548a481014b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:17.146195 23269 raft_consensus.cc:729] T ddb6fd5c5f6e42cdbc4f615042249fbb P 5b2c000f19594cfa8696548a481014b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b2c000f19594cfa8696548a481014b7, State: Initialized, Role: FOLLOWER
I20260806 11:36:17.146507 22724 catalog_manager.cc:4841] TS 5b2c000f19594cfa8696548a481014b7 (127.21.14.193:40535): tablet c2e92516c2294258a2e3d8cb31249655 (table test-table [id=dfb301a9dfd94b63980c94ba92f17522]) successfully deleted
I20260806 11:36:17.147256 23269 consensus_queue.cc:260] T ddb6fd5c5f6e42cdbc4f615042249fbb P 5b2c000f19594cfa8696548a481014b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2c000f19594cfa8696548a481014b7" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40535 } } peers { permanent_uuid: "f32715fb8fff45afb9e03ef09c7eb305" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 40175 } } peers { permanent_uuid: "ce59d498c5d045519b0d4f5306c4e46d" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 36341 } }
I20260806 11:36:17.147704 23269 ts_tablet_manager.cc:1428] T ddb6fd5c5f6e42cdbc4f615042249fbb P 5b2c000f19594cfa8696548a481014b7: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 11:36:17.148326 22722 catalog_manager.cc:4841] TS f32715fb8fff45afb9e03ef09c7eb305 (127.21.14.194:40175): tablet c2e92516c2294258a2e3d8cb31249655 (table test-table [id=dfb301a9dfd94b63980c94ba92f17522]) successfully deleted
I20260806 11:36:17.148841 23271 tablet_bootstrap.cc:492] T ddb6fd5c5f6e42cdbc4f615042249fbb P f32715fb8fff45afb9e03ef09c7eb305: No bootstrap required, opened a new log
I20260806 11:36:17.148938 23271 ts_tablet_manager.cc:1397] T ddb6fd5c5f6e42cdbc4f615042249fbb P f32715fb8fff45afb9e03ef09c7eb305: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260806 11:36:17.149446 23271 raft_consensus.cc:348] T ddb6fd5c5f6e42cdbc4f615042249fbb P f32715fb8fff45afb9e03ef09c7eb305 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2c000f19594cfa8696548a481014b7" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40535 } } peers { permanent_uuid: "f32715fb8fff45afb9e03ef09c7eb305" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 40175 } } peers { permanent_uuid: "ce59d498c5d045519b0d4f5306c4e46d" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 36341 } }
I20260806 11:36:17.149585 23271 raft_consensus.cc:374] T ddb6fd5c5f6e42cdbc4f615042249fbb P f32715fb8fff45afb9e03ef09c7eb305 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:17.149617 23271 raft_consensus.cc:729] T ddb6fd5c5f6e42cdbc4f615042249fbb P f32715fb8fff45afb9e03ef09c7eb305 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f32715fb8fff45afb9e03ef09c7eb305, State: Initialized, Role: FOLLOWER
I20260806 11:36:17.149756 23271 consensus_queue.cc:260] T ddb6fd5c5f6e42cdbc4f615042249fbb P f32715fb8fff45afb9e03ef09c7eb305 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2c000f19594cfa8696548a481014b7" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40535 } } peers { permanent_uuid: "f32715fb8fff45afb9e03ef09c7eb305" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 40175 } } peers { permanent_uuid: "ce59d498c5d045519b0d4f5306c4e46d" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 36341 } }
I20260806 11:36:17.150019 23271 ts_tablet_manager.cc:1428] T ddb6fd5c5f6e42cdbc4f615042249fbb P f32715fb8fff45afb9e03ef09c7eb305: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 11:36:17.426201 23276 raft_consensus.cc:482] T ddb6fd5c5f6e42cdbc4f615042249fbb P ce59d498c5d045519b0d4f5306c4e46d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 11:36:17.426383 23276 raft_consensus.cc:504] T ddb6fd5c5f6e42cdbc4f615042249fbb P ce59d498c5d045519b0d4f5306c4e46d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2c000f19594cfa8696548a481014b7" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40535 } } peers { permanent_uuid: "f32715fb8fff45afb9e03ef09c7eb305" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 40175 } } peers { permanent_uuid: "ce59d498c5d045519b0d4f5306c4e46d" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 36341 } }
I20260806 11:36:17.427644 23276 leader_election.cc:288] T ddb6fd5c5f6e42cdbc4f615042249fbb P ce59d498c5d045519b0d4f5306c4e46d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b2c000f19594cfa8696548a481014b7 (127.21.14.193:40535), f32715fb8fff45afb9e03ef09c7eb305 (127.21.14.194:40175)
I20260806 11:36:17.431988 23263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:17.428089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:59325 (local address 127.21.14.194:40175)
0806 11:36:17.428228 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:17.428233 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:17.428592 (+ 359us) server_negotiation.cc:407] Connection header received
0806 11:36:17.428747 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:17.428750 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:17.428806 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:17.428885 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:36:17.429159 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:17.429679 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:17.430820 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:17.431313 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:17.431451 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:17.431518 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:17.431744 (+ 226us) server_negotiation.cc:299] Negotiation successful
0806 11:36:17.431831 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 11:36:17.432006 23300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:17.427923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.193:40535 (local address 127.21.14.195:33851)
0806 11:36:17.428303 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:17.428315 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:17.428429 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:17.428726 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:17.428730 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:17.428750 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:36:17.428898 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:17.428905 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:17.429719 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:17.429725 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:17.430688 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:17.430698 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:17.431452 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:17.431456 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:17.431631 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:17.431650 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 11:36:17.431742 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 11:36:17.431823 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"mutex_wait_us":72,"thread_start_us":115,"threads_started":1}
I20260806 11:36:17.431988 23267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:17.428010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:33851 (local address 127.21.14.193:40535)
0806 11:36:17.428194 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:17.428202 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:17.428414 (+ 212us) server_negotiation.cc:407] Connection header received
0806 11:36:17.428536 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:17.428540 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:17.428602 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:17.428688 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:36:17.429014 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:17.429593 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:17.430826 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:17.431312 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:17.431452 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:17.431518 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:17.431776 (+ 258us) server_negotiation.cc:299] Negotiation successful
0806 11:36:17.431865 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260806 11:36:17.432006 23301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:17.427964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.194:40175 (local address 127.21.14.195:59325)
0806 11:36:17.428476 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:17.428492 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:17.428626 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:17.428907 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:17.428909 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:17.428924 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:36:17.429047 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:17.429053 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:17.429793 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:17.429795 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:17.430683 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:17.430694 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:17.431452 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:17.431456 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:17.431619 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:17.431640 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 11:36:17.431703 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 11:36:17.431812 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"mutex_wait_us":31,"thread_start_us":79,"threads_started":1}
I20260806 11:36:17.432505 22902 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ddb6fd5c5f6e42cdbc4f615042249fbb" candidate_uuid: "ce59d498c5d045519b0d4f5306c4e46d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b2c000f19594cfa8696548a481014b7" is_pre_election: true
I20260806 11:36:17.432674 22902 raft_consensus.cc:2454] T ddb6fd5c5f6e42cdbc4f615042249fbb P 5b2c000f19594cfa8696548a481014b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce59d498c5d045519b0d4f5306c4e46d in term 0.
I20260806 11:36:17.432622 23050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ddb6fd5c5f6e42cdbc4f615042249fbb" candidate_uuid: "ce59d498c5d045519b0d4f5306c4e46d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f32715fb8fff45afb9e03ef09c7eb305" is_pre_election: true
I20260806 11:36:17.432893 23050 raft_consensus.cc:2454] T ddb6fd5c5f6e42cdbc4f615042249fbb P f32715fb8fff45afb9e03ef09c7eb305 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce59d498c5d045519b0d4f5306c4e46d in term 0.
I20260806 11:36:17.433079 23128 leader_election.cc:302] T ddb6fd5c5f6e42cdbc4f615042249fbb P ce59d498c5d045519b0d4f5306c4e46d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b2c000f19594cfa8696548a481014b7, ce59d498c5d045519b0d4f5306c4e46d; no voters:
I20260806 11:36:17.433418 23276 raft_consensus.cc:2781] T ddb6fd5c5f6e42cdbc4f615042249fbb P ce59d498c5d045519b0d4f5306c4e46d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 11:36:17.433521 23276 raft_consensus.cc:482] T ddb6fd5c5f6e42cdbc4f615042249fbb P ce59d498c5d045519b0d4f5306c4e46d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 11:36:17.433571 23276 raft_consensus.cc:3037] T ddb6fd5c5f6e42cdbc4f615042249fbb P ce59d498c5d045519b0d4f5306c4e46d [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:36:17.434705 23276 raft_consensus.cc:504] T ddb6fd5c5f6e42cdbc4f615042249fbb P ce59d498c5d045519b0d4f5306c4e46d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2c000f19594cfa8696548a481014b7" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40535 } } peers { permanent_uuid: "f32715fb8fff45afb9e03ef09c7eb305" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 40175 } } peers { permanent_uuid: "ce59d498c5d045519b0d4f5306c4e46d" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 36341 } }
I20260806 11:36:17.435190 23276 leader_election.cc:288] T ddb6fd5c5f6e42cdbc4f615042249fbb P ce59d498c5d045519b0d4f5306c4e46d [CANDIDATE]: Term 1 election: Requested vote from peers 5b2c000f19594cfa8696548a481014b7 (127.21.14.193:40535), f32715fb8fff45afb9e03ef09c7eb305 (127.21.14.194:40175)
I20260806 11:36:17.435932 23050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ddb6fd5c5f6e42cdbc4f615042249fbb" candidate_uuid: "ce59d498c5d045519b0d4f5306c4e46d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f32715fb8fff45afb9e03ef09c7eb305"
I20260806 11:36:17.435824 22902 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ddb6fd5c5f6e42cdbc4f615042249fbb" candidate_uuid: "ce59d498c5d045519b0d4f5306c4e46d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b2c000f19594cfa8696548a481014b7"
I20260806 11:36:17.436100 23050 raft_consensus.cc:3037] T ddb6fd5c5f6e42cdbc4f615042249fbb P f32715fb8fff45afb9e03ef09c7eb305 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:36:17.436116 22902 raft_consensus.cc:3037] T ddb6fd5c5f6e42cdbc4f615042249fbb P 5b2c000f19594cfa8696548a481014b7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:36:17.437181 22902 raft_consensus.cc:2454] T ddb6fd5c5f6e42cdbc4f615042249fbb P 5b2c000f19594cfa8696548a481014b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce59d498c5d045519b0d4f5306c4e46d in term 1.
I20260806 11:36:17.437187 23050 raft_consensus.cc:2454] T ddb6fd5c5f6e42cdbc4f615042249fbb P f32715fb8fff45afb9e03ef09c7eb305 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce59d498c5d045519b0d4f5306c4e46d in term 1.
I20260806 11:36:17.437565 23128 leader_election.cc:302] T ddb6fd5c5f6e42cdbc4f615042249fbb P ce59d498c5d045519b0d4f5306c4e46d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b2c000f19594cfa8696548a481014b7, ce59d498c5d045519b0d4f5306c4e46d; no voters:
I20260806 11:36:17.437776 23276 raft_consensus.cc:2781] T ddb6fd5c5f6e42cdbc4f615042249fbb P ce59d498c5d045519b0d4f5306c4e46d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:36:17.437983 23276 raft_consensus.cc:686] T ddb6fd5c5f6e42cdbc4f615042249fbb P ce59d498c5d045519b0d4f5306c4e46d [term 1 LEADER]: Becoming Leader. State: Replica: ce59d498c5d045519b0d4f5306c4e46d, State: Running, Role: LEADER
I20260806 11:36:17.438269 23276 consensus_queue.cc:237] T ddb6fd5c5f6e42cdbc4f615042249fbb P ce59d498c5d045519b0d4f5306c4e46d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2c000f19594cfa8696548a481014b7" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40535 } } peers { permanent_uuid: "f32715fb8fff45afb9e03ef09c7eb305" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 40175 } } peers { permanent_uuid: "ce59d498c5d045519b0d4f5306c4e46d" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 36341 } }
I20260806 11:36:17.440745 22734 catalog_manager.cc:5495] T ddb6fd5c5f6e42cdbc4f615042249fbb P ce59d498c5d045519b0d4f5306c4e46d reported cstate change: term changed from 0 to 1, leader changed from <none> to ce59d498c5d045519b0d4f5306c4e46d (127.21.14.195). New cstate: current_term: 1 leader_uuid: "ce59d498c5d045519b0d4f5306c4e46d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2c000f19594cfa8696548a481014b7" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f32715fb8fff45afb9e03ef09c7eb305" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 40175 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce59d498c5d045519b0d4f5306c4e46d" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 36341 } health_report { overall_health: HEALTHY } } }
Aug 06 11:36:17 dist-test-slave-7c35 krb5kdc[22690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016175, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.254@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:36:17.483286 23312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:17.473012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:40925 (local address 127.0.0.1:50358)
0806 11:36:17.473453 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:17.473467 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:17.473629 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:17.474059 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:17.474062 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:17.474555 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:17.474693 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:17.474768 (+ 75us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:17.475888 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:17.475891 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:17.476472 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:17.476480 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:17.477059 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:17.477063 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:17.477145 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:17.477855 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:17.477869 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:17.479519 (+ 1650us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:17.481599 (+ 2080us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:17.481612 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:17.481620 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:17.481889 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:17.482161 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:17.482163 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:17.482165 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:17.482243 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:17.482638 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:17.482645 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:17.482766 (+ 121us) client_negotiation.cc:769] Sending connection context
0806 11:36:17.482961 (+ 195us) client_negotiation.cc:240] Negotiation successful
0806 11:36:17.483131 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":176,"threads_started":1}
I20260806 11:36:17.483418 23311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:17.473114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50358 (local address 127.21.14.254:40925)
0806 11:36:17.473444 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:17.473455 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:17.473570 (+ 115us) server_negotiation.cc:407] Connection header received
0806 11:36:17.473765 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:17.473769 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:17.473834 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:17.473939 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:17.475121 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:17.475727 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:17.476619 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:17.476908 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:17.477018 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:17.479706 (+ 2688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:17.479713 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:17.479716 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:17.479751 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:17.481468 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:17.481995 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:17.481997 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:17.481999 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:17.482082 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:17.482318 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:17.482320 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:17.482321 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:17.482545 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:17.482622 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:17.483018 (+ 396us) server_negotiation.cc:299] Negotiation successful
0806 11:36:17.483229 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":66,"server-negotiator.queue_time_us":239,"thread_start_us":118,"threads_started":1}
I20260806 11:36:17.488085 22734 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.21.14.254@KRBTEST.COM'} at 127.0.0.1:50358:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 11:36:17.488993 22734 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 11:36:17.494702 22881 tablet_service.cc:1467] Processing CreateTablet for tablet b0dacb6de936447f8c3e49f7b891a559 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a958cfc8358f4384bc39dd12639cb8f0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 11:36:17.494694 23029 tablet_service.cc:1467] Processing CreateTablet for tablet b0dacb6de936447f8c3e49f7b891a559 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a958cfc8358f4384bc39dd12639cb8f0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 11:36:17.494767 23176 tablet_service.cc:1467] Processing CreateTablet for tablet b0dacb6de936447f8c3e49f7b891a559 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a958cfc8358f4384bc39dd12639cb8f0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 11:36:17.495270 23029 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0dacb6de936447f8c3e49f7b891a559. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:17.495270 23176 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0dacb6de936447f8c3e49f7b891a559. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:17.495271 22881 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0dacb6de936447f8c3e49f7b891a559. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:17.498142 23269 tablet_bootstrap.cc:492] T b0dacb6de936447f8c3e49f7b891a559 P 5b2c000f19594cfa8696548a481014b7: Bootstrap starting.
I20260806 11:36:17.498198 23271 tablet_bootstrap.cc:492] T b0dacb6de936447f8c3e49f7b891a559 P f32715fb8fff45afb9e03ef09c7eb305: Bootstrap starting.
I20260806 11:36:17.498240 23270 tablet_bootstrap.cc:492] T b0dacb6de936447f8c3e49f7b891a559 P ce59d498c5d045519b0d4f5306c4e46d: Bootstrap starting.
I20260806 11:36:17.499256 23269 tablet_bootstrap.cc:654] T b0dacb6de936447f8c3e49f7b891a559 P 5b2c000f19594cfa8696548a481014b7: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:17.499256 23271 tablet_bootstrap.cc:654] T b0dacb6de936447f8c3e49f7b891a559 P f32715fb8fff45afb9e03ef09c7eb305: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:17.499354 23270 tablet_bootstrap.cc:654] T b0dacb6de936447f8c3e49f7b891a559 P ce59d498c5d045519b0d4f5306c4e46d: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:17.500726 23269 tablet_bootstrap.cc:492] T b0dacb6de936447f8c3e49f7b891a559 P 5b2c000f19594cfa8696548a481014b7: No bootstrap required, opened a new log
I20260806 11:36:17.500741 23270 tablet_bootstrap.cc:492] T b0dacb6de936447f8c3e49f7b891a559 P ce59d498c5d045519b0d4f5306c4e46d: No bootstrap required, opened a new log
I20260806 11:36:17.500775 23271 tablet_bootstrap.cc:492] T b0dacb6de936447f8c3e49f7b891a559 P f32715fb8fff45afb9e03ef09c7eb305: No bootstrap required, opened a new log
I20260806 11:36:17.500844 23271 ts_tablet_manager.cc:1397] T b0dacb6de936447f8c3e49f7b891a559 P f32715fb8fff45afb9e03ef09c7eb305: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 11:36:17.500844 23269 ts_tablet_manager.cc:1397] T b0dacb6de936447f8c3e49f7b891a559 P 5b2c000f19594cfa8696548a481014b7: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 11:36:17.500838 23270 ts_tablet_manager.cc:1397] T b0dacb6de936447f8c3e49f7b891a559 P ce59d498c5d045519b0d4f5306c4e46d: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260806 11:36:17.501292 23270 raft_consensus.cc:348] T b0dacb6de936447f8c3e49f7b891a559 P ce59d498c5d045519b0d4f5306c4e46d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2c000f19594cfa8696548a481014b7" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40535 } } peers { permanent_uuid: "f32715fb8fff45afb9e03ef09c7eb305" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 40175 } } peers { permanent_uuid: "ce59d498c5d045519b0d4f5306c4e46d" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 36341 } }
I20260806 11:36:17.501292 23271 raft_consensus.cc:348] T b0dacb6de936447f8c3e49f7b891a559 P f32715fb8fff45afb9e03ef09c7eb305 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2c000f19594cfa8696548a481014b7" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40535 } } peers { permanent_uuid: "f32715fb8fff45afb9e03ef09c7eb305" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 40175 } } peers { permanent_uuid: "ce59d498c5d045519b0d4f5306c4e46d" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 36341 } }
I20260806 11:36:17.501333 23269 raft_consensus.cc:348] T b0dacb6de936447f8c3e49f7b891a559 P 5b2c000f19594cfa8696548a481014b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2c000f19594cfa8696548a481014b7" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40535 } } peers { permanent_uuid: "f32715fb8fff45afb9e03ef09c7eb305" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 40175 } } peers { permanent_uuid: "ce59d498c5d045519b0d4f5306c4e46d" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 36341 } }
I20260806 11:36:17.501422 23271 raft_consensus.cc:374] T b0dacb6de936447f8c3e49f7b891a559 P f32715fb8fff45afb9e03ef09c7eb305 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:17.501423 23270 raft_consensus.cc:374] T b0dacb6de936447f8c3e49f7b891a559 P ce59d498c5d045519b0d4f5306c4e46d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:17.501449 23269 raft_consensus.cc:374] T b0dacb6de936447f8c3e49f7b891a559 P 5b2c000f19594cfa8696548a481014b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:17.501461 23271 raft_consensus.cc:729] T b0dacb6de936447f8c3e49f7b891a559 P f32715fb8fff45afb9e03ef09c7eb305 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f32715fb8fff45afb9e03ef09c7eb305, State: Initialized, Role: FOLLOWER
I20260806 11:36:17.501461 23270 raft_consensus.cc:729] T b0dacb6de936447f8c3e49f7b891a559 P ce59d498c5d045519b0d4f5306c4e46d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce59d498c5d045519b0d4f5306c4e46d, State: Initialized, Role: FOLLOWER
I20260806 11:36:17.501495 23269 raft_consensus.cc:729] T b0dacb6de936447f8c3e49f7b891a559 P 5b2c000f19594cfa8696548a481014b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b2c000f19594cfa8696548a481014b7, State: Initialized, Role: FOLLOWER
I20260806 11:36:17.501603 23270 consensus_queue.cc:260] T b0dacb6de936447f8c3e49f7b891a559 P ce59d498c5d045519b0d4f5306c4e46d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2c000f19594cfa8696548a481014b7" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40535 } } peers { permanent_uuid: "f32715fb8fff45afb9e03ef09c7eb305" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 40175 } } peers { permanent_uuid: "ce59d498c5d045519b0d4f5306c4e46d" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 36341 } }
I20260806 11:36:17.501592 23271 consensus_queue.cc:260] T b0dacb6de936447f8c3e49f7b891a559 P f32715fb8fff45afb9e03ef09c7eb305 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2c000f19594cfa8696548a481014b7" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40535 } } peers { permanent_uuid: "f32715fb8fff45afb9e03ef09c7eb305" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 40175 } } peers { permanent_uuid: "ce59d498c5d045519b0d4f5306c4e46d" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 36341 } }
I20260806 11:36:17.501626 23269 consensus_queue.cc:260] T b0dacb6de936447f8c3e49f7b891a559 P 5b2c000f19594cfa8696548a481014b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2c000f19594cfa8696548a481014b7" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40535 } } peers { permanent_uuid: "f32715fb8fff45afb9e03ef09c7eb305" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 40175 } } peers { permanent_uuid: "ce59d498c5d045519b0d4f5306c4e46d" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 36341 } }
I20260806 11:36:17.501881 23271 ts_tablet_manager.cc:1428] T b0dacb6de936447f8c3e49f7b891a559 P f32715fb8fff45afb9e03ef09c7eb305: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 11:36:17.501891 23270 ts_tablet_manager.cc:1428] T b0dacb6de936447f8c3e49f7b891a559 P ce59d498c5d045519b0d4f5306c4e46d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 11:36:17.502013 23275 tablet_replica.cc:439] T b0dacb6de936447f8c3e49f7b891a559 P 5b2c000f19594cfa8696548a481014b7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2c000f19594cfa8696548a481014b7" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40535 } } peers { permanent_uuid: "f32715fb8fff45afb9e03ef09c7eb305" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 40175 } } peers { permanent_uuid: "ce59d498c5d045519b0d4f5306c4e46d" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 36341 } } }
I20260806 11:36:17.502115 23276 tablet_replica.cc:439] T b0dacb6de936447f8c3e49f7b891a559 P ce59d498c5d045519b0d4f5306c4e46d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2c000f19594cfa8696548a481014b7" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40535 } } peers { permanent_uuid: "f32715fb8fff45afb9e03ef09c7eb305" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 40175 } } peers { permanent_uuid: "ce59d498c5d045519b0d4f5306c4e46d" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 36341 } } }
I20260806 11:36:17.502198 23275 tablet_replica.cc:442] T b0dacb6de936447f8c3e49f7b891a559 P 5b2c000f19594cfa8696548a481014b7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 11:36:17.502116 23291 tablet_replica.cc:439] T b0dacb6de936447f8c3e49f7b891a559 P f32715fb8fff45afb9e03ef09c7eb305: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2c000f19594cfa8696548a481014b7" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40535 } } peers { permanent_uuid: "f32715fb8fff45afb9e03ef09c7eb305" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 40175 } } peers { permanent_uuid: "ce59d498c5d045519b0d4f5306c4e46d" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 36341 } } }
I20260806 11:36:17.502247 23276 tablet_replica.cc:442] T b0dacb6de936447f8c3e49f7b891a559 P ce59d498c5d045519b0d4f5306c4e46d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 11:36:17.502260 23291 tablet_replica.cc:442] T b0dacb6de936447f8c3e49f7b891a559 P f32715fb8fff45afb9e03ef09c7eb305: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 11:36:17.502018 23269 ts_tablet_manager.cc:1428] T b0dacb6de936447f8c3e49f7b891a559 P 5b2c000f19594cfa8696548a481014b7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 11:36:17.702739 23275 raft_consensus.cc:482] T b0dacb6de936447f8c3e49f7b891a559 P 5b2c000f19594cfa8696548a481014b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 11:36:17.703091 23275 raft_consensus.cc:504] T b0dacb6de936447f8c3e49f7b891a559 P 5b2c000f19594cfa8696548a481014b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2c000f19594cfa8696548a481014b7" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40535 } } peers { permanent_uuid: "f32715fb8fff45afb9e03ef09c7eb305" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 40175 } } peers { permanent_uuid: "ce59d498c5d045519b0d4f5306c4e46d" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 36341 } }
I20260806 11:36:17.704465 23275 leader_election.cc:288] T b0dacb6de936447f8c3e49f7b891a559 P 5b2c000f19594cfa8696548a481014b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f32715fb8fff45afb9e03ef09c7eb305 (127.21.14.194:40175), ce59d498c5d045519b0d4f5306c4e46d (127.21.14.195:36341)
I20260806 11:36:17.709532 23316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:17.704734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.195:36341 (local address 127.21.14.193:35281)
0806 11:36:17.705148 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:17.705163 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:17.705275 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:17.705816 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:17.705818 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:17.705843 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:36:17.706020 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:17.706027 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:17.707016 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:17.707021 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:17.708016 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:17.708028 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:17.708946 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:17.708973 (+ 27us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:17.709088 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:17.709120 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 11:36:17.709234 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 11:36:17.709329 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":99,"threads_started":1}
I20260806 11:36:17.709601 23318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:17.704912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:35281 (local address 127.21.14.195:36341)
0806 11:36:17.705275 (+ 363us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:17.705283 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:17.705318 (+ 35us) server_negotiation.cc:407] Connection header received
0806 11:36:17.705441 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:17.705445 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:17.705540 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:17.705819 (+ 279us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:36:17.706185 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:17.706896 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:17.708166 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:17.708704 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:17.708816 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:17.708864 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:17.709307 (+ 443us) server_negotiation.cc:299] Negotiation successful
0806 11:36:17.709448 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":62,"threads_started":1}
I20260806 11:36:17.710068 23263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:17.704833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:59653 (local address 127.21.14.194:40175)
0806 11:36:17.705263 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:17.705274 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:17.705809 (+ 535us) server_negotiation.cc:407] Connection header received
0806 11:36:17.705946 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:17.705955 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:17.706013 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:17.706108 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:36:17.706429 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:17.707016 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:17.708312 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:17.709193 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:17.709345 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:17.709411 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:17.709816 (+ 405us) server_negotiation.cc:299] Negotiation successful
0806 11:36:17.709945 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208}
I20260806 11:36:17.710280 23196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0dacb6de936447f8c3e49f7b891a559" candidate_uuid: "5b2c000f19594cfa8696548a481014b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce59d498c5d045519b0d4f5306c4e46d" is_pre_election: true
I20260806 11:36:17.710487 23196 raft_consensus.cc:2454] T b0dacb6de936447f8c3e49f7b891a559 P ce59d498c5d045519b0d4f5306c4e46d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b2c000f19594cfa8696548a481014b7 in term 0.
I20260806 11:36:17.710564 23317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:17.704735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.194:40175 (local address 127.21.14.193:59653)
0806 11:36:17.705587 (+ 852us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:17.705701 (+ 114us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:17.705830 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:17.706170 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:17.706174 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:17.706194 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:36:17.706301 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:17.706308 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:17.707152 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:17.707155 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:17.708075 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:17.708110 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:17.709391 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:17.709395 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:17.709541 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:17.709566 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 11:36:17.710267 (+ 701us) client_negotiation.cc:240] Negotiation successful
0806 11:36:17.710360 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":717,"mutex_wait_us":27,"thread_start_us":71,"threads_started":1}
I20260806 11:36:17.711005 22830 leader_election.cc:302] T b0dacb6de936447f8c3e49f7b891a559 P 5b2c000f19594cfa8696548a481014b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b2c000f19594cfa8696548a481014b7, ce59d498c5d045519b0d4f5306c4e46d; no voters:
I20260806 11:36:17.711084 23050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0dacb6de936447f8c3e49f7b891a559" candidate_uuid: "5b2c000f19594cfa8696548a481014b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f32715fb8fff45afb9e03ef09c7eb305" is_pre_election: true
I20260806 11:36:17.711265 23050 raft_consensus.cc:2454] T b0dacb6de936447f8c3e49f7b891a559 P f32715fb8fff45afb9e03ef09c7eb305 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b2c000f19594cfa8696548a481014b7 in term 0.
I20260806 11:36:17.711309 23275 raft_consensus.cc:2781] T b0dacb6de936447f8c3e49f7b891a559 P 5b2c000f19594cfa8696548a481014b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 11:36:17.711364 23275 raft_consensus.cc:482] T b0dacb6de936447f8c3e49f7b891a559 P 5b2c000f19594cfa8696548a481014b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 11:36:17.711405 23275 raft_consensus.cc:3037] T b0dacb6de936447f8c3e49f7b891a559 P 5b2c000f19594cfa8696548a481014b7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:36:17.712474 23275 raft_consensus.cc:504] T b0dacb6de936447f8c3e49f7b891a559 P 5b2c000f19594cfa8696548a481014b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2c000f19594cfa8696548a481014b7" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40535 } } peers { permanent_uuid: "f32715fb8fff45afb9e03ef09c7eb305" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 40175 } } peers { permanent_uuid: "ce59d498c5d045519b0d4f5306c4e46d" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 36341 } }
I20260806 11:36:17.712879 23275 leader_election.cc:288] T b0dacb6de936447f8c3e49f7b891a559 P 5b2c000f19594cfa8696548a481014b7 [CANDIDATE]: Term 1 election: Requested vote from peers f32715fb8fff45afb9e03ef09c7eb305 (127.21.14.194:40175), ce59d498c5d045519b0d4f5306c4e46d (127.21.14.195:36341)
I20260806 11:36:17.713369 23050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0dacb6de936447f8c3e49f7b891a559" candidate_uuid: "5b2c000f19594cfa8696548a481014b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f32715fb8fff45afb9e03ef09c7eb305"
I20260806 11:36:17.713562 23050 raft_consensus.cc:3037] T b0dacb6de936447f8c3e49f7b891a559 P f32715fb8fff45afb9e03ef09c7eb305 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:36:17.714645 23050 raft_consensus.cc:2454] T b0dacb6de936447f8c3e49f7b891a559 P f32715fb8fff45afb9e03ef09c7eb305 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b2c000f19594cfa8696548a481014b7 in term 1.
I20260806 11:36:17.715015 23196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0dacb6de936447f8c3e49f7b891a559" candidate_uuid: "5b2c000f19594cfa8696548a481014b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce59d498c5d045519b0d4f5306c4e46d"
I20260806 11:36:17.715167 23196 raft_consensus.cc:3037] T b0dacb6de936447f8c3e49f7b891a559 P ce59d498c5d045519b0d4f5306c4e46d [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:36:17.715063 22832 leader_election.cc:302] T b0dacb6de936447f8c3e49f7b891a559 P 5b2c000f19594cfa8696548a481014b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b2c000f19594cfa8696548a481014b7, f32715fb8fff45afb9e03ef09c7eb305; no voters:
I20260806 11:36:17.715395 23275 raft_consensus.cc:2781] T b0dacb6de936447f8c3e49f7b891a559 P 5b2c000f19594cfa8696548a481014b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:36:17.715601 23275 raft_consensus.cc:686] T b0dacb6de936447f8c3e49f7b891a559 P 5b2c000f19594cfa8696548a481014b7 [term 1 LEADER]: Becoming Leader. State: Replica: 5b2c000f19594cfa8696548a481014b7, State: Running, Role: LEADER
I20260806 11:36:17.715900 23275 consensus_queue.cc:237] T b0dacb6de936447f8c3e49f7b891a559 P 5b2c000f19594cfa8696548a481014b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2c000f19594cfa8696548a481014b7" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40535 } } peers { permanent_uuid: "f32715fb8fff45afb9e03ef09c7eb305" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 40175 } } peers { permanent_uuid: "ce59d498c5d045519b0d4f5306c4e46d" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 36341 } }
I20260806 11:36:17.716349 23196 raft_consensus.cc:2454] T b0dacb6de936447f8c3e49f7b891a559 P ce59d498c5d045519b0d4f5306c4e46d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b2c000f19594cfa8696548a481014b7 in term 1.
I20260806 11:36:17.717676 23319 tablet_replica.cc:439] T b0dacb6de936447f8c3e49f7b891a559 P 5b2c000f19594cfa8696548a481014b7: TxnStatusTablet state changed. Reason: New leader 5b2c000f19594cfa8696548a481014b7. Latest consensus state: current_term: 1 leader_uuid: "5b2c000f19594cfa8696548a481014b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2c000f19594cfa8696548a481014b7" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40535 } } peers { permanent_uuid: "f32715fb8fff45afb9e03ef09c7eb305" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 40175 } } peers { permanent_uuid: "ce59d498c5d045519b0d4f5306c4e46d" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 36341 } } }
I20260806 11:36:17.717823 23319 tablet_replica.cc:442] T b0dacb6de936447f8c3e49f7b891a559 P 5b2c000f19594cfa8696548a481014b7: This TxnStatusTablet replica's current role is: LEADER
I20260806 11:36:17.718194 23321 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 11:36:17.718428 23321 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 11:36:17.719456 22736 catalog_manager.cc:5495] T b0dacb6de936447f8c3e49f7b891a559 P 5b2c000f19594cfa8696548a481014b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5b2c000f19594cfa8696548a481014b7 (127.21.14.193). New cstate: current_term: 1 leader_uuid: "5b2c000f19594cfa8696548a481014b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2c000f19594cfa8696548a481014b7" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40535 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f32715fb8fff45afb9e03ef09c7eb305" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 40175 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce59d498c5d045519b0d4f5306c4e46d" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 36341 } health_report { overall_health: UNKNOWN } } }
I20260806 11:36:17.751356 23312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:17.746356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.193:40535 (local address 127.0.0.1:46570)
0806 11:36:17.746654 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:17.746714 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:17.746905 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:17.747383 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:17.747386 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:17.747393 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 11:36:17.747508 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:17.747513 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:17.748622 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:17.748626 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:17.749358 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:17.749370 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:17.749987 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:17.749989 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:17.750072 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:17.750143 (+ 71us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 11:36:17.751060 (+ 917us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 11:36:17.751079 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 11:36:17.751138 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 11:36:17.751227 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151}
I20260806 11:36:17.751370 23267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:17.746583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46570 (local address 127.21.14.193:40535)
0806 11:36:17.746737 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:17.746747 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:17.746770 (+ 23us) server_negotiation.cc:407] Connection header received
0806 11:36:17.747110 (+ 340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:17.747113 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:17.747217 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:17.747330 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 11:36:17.747627 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:17.748440 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:17.749489 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:17.749845 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:17.749957 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:17.750295 (+ 338us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 11:36:17.750964 (+ 669us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 11:36:17.751078 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:17.751151 (+ 73us) server_negotiation.cc:299] Negotiation successful
0806 11:36:17.751221 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 11:36:17.755132 23196 raft_consensus.cc:1261] T b0dacb6de936447f8c3e49f7b891a559 P ce59d498c5d045519b0d4f5306c4e46d [term 1 FOLLOWER]: Refusing update from remote peer 5b2c000f19594cfa8696548a481014b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 11:36:17.755438 23050 raft_consensus.cc:1261] T b0dacb6de936447f8c3e49f7b891a559 P f32715fb8fff45afb9e03ef09c7eb305 [term 1 FOLLOWER]: Refusing update from remote peer 5b2c000f19594cfa8696548a481014b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 11:36:17.755694 23319 consensus_queue.cc:1035] T b0dacb6de936447f8c3e49f7b891a559 P 5b2c000f19594cfa8696548a481014b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce59d498c5d045519b0d4f5306c4e46d" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 36341 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 11:36:17.756011 23275 consensus_queue.cc:1035] T b0dacb6de936447f8c3e49f7b891a559 P 5b2c000f19594cfa8696548a481014b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f32715fb8fff45afb9e03ef09c7eb305" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 40175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 11:36:17.757725 23276 tablet_replica.cc:439] T b0dacb6de936447f8c3e49f7b891a559 P ce59d498c5d045519b0d4f5306c4e46d: TxnStatusTablet state changed. Reason: New leader 5b2c000f19594cfa8696548a481014b7. Latest consensus state: current_term: 1 leader_uuid: "5b2c000f19594cfa8696548a481014b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2c000f19594cfa8696548a481014b7" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40535 } } peers { permanent_uuid: "f32715fb8fff45afb9e03ef09c7eb305" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 40175 } } peers { permanent_uuid: "ce59d498c5d045519b0d4f5306c4e46d" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 36341 } } }
I20260806 11:36:17.757854 23276 tablet_replica.cc:442] T b0dacb6de936447f8c3e49f7b891a559 P ce59d498c5d045519b0d4f5306c4e46d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 11:36:17.757921 23291 tablet_replica.cc:439] T b0dacb6de936447f8c3e49f7b891a559 P f32715fb8fff45afb9e03ef09c7eb305: TxnStatusTablet state changed. Reason: New leader 5b2c000f19594cfa8696548a481014b7. Latest consensus state: current_term: 1 leader_uuid: "5b2c000f19594cfa8696548a481014b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2c000f19594cfa8696548a481014b7" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40535 } } peers { permanent_uuid: "f32715fb8fff45afb9e03ef09c7eb305" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 40175 } } peers { permanent_uuid: "ce59d498c5d045519b0d4f5306c4e46d" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 36341 } } }
I20260806 11:36:17.758059 23291 tablet_replica.cc:442] T b0dacb6de936447f8c3e49f7b891a559 P f32715fb8fff45afb9e03ef09c7eb305: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 11:36:17.760856 23319 tablet_replica.cc:439] T b0dacb6de936447f8c3e49f7b891a559 P 5b2c000f19594cfa8696548a481014b7: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5b2c000f19594cfa8696548a481014b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2c000f19594cfa8696548a481014b7" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40535 } } peers { permanent_uuid: "f32715fb8fff45afb9e03ef09c7eb305" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 40175 } } peers { permanent_uuid: "ce59d498c5d045519b0d4f5306c4e46d" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 36341 } } }
I20260806 11:36:17.761008 23319 tablet_replica.cc:442] T b0dacb6de936447f8c3e49f7b891a559 P 5b2c000f19594cfa8696548a481014b7: This TxnStatusTablet replica's current role is: LEADER
I20260806 11:36:17.760879 23275 tablet_replica.cc:439] T b0dacb6de936447f8c3e49f7b891a559 P 5b2c000f19594cfa8696548a481014b7: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5b2c000f19594cfa8696548a481014b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2c000f19594cfa8696548a481014b7" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40535 } } peers { permanent_uuid: "f32715fb8fff45afb9e03ef09c7eb305" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 40175 } } peers { permanent_uuid: "ce59d498c5d045519b0d4f5306c4e46d" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 36341 } } }
I20260806 11:36:17.761760 23275 tablet_replica.cc:442] T b0dacb6de936447f8c3e49f7b891a559 P 5b2c000f19594cfa8696548a481014b7: This TxnStatusTablet replica's current role is: LEADER
I20260806 11:36:17.762682 23325 mvcc.cc:205] Tried to move back new op lower bound from 7315522264078766080 to 7315522263927640064. Current Snapshot: MvccSnapshot[applied={T|T < 7315522264078766080}]
I20260806 11:36:17.763015 23276 tablet_replica.cc:439] T b0dacb6de936447f8c3e49f7b891a559 P ce59d498c5d045519b0d4f5306c4e46d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5b2c000f19594cfa8696548a481014b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2c000f19594cfa8696548a481014b7" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40535 } } peers { permanent_uuid: "f32715fb8fff45afb9e03ef09c7eb305" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 40175 } } peers { permanent_uuid: "ce59d498c5d045519b0d4f5306c4e46d" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 36341 } } }
I20260806 11:36:17.763131 23276 tablet_replica.cc:442] T b0dacb6de936447f8c3e49f7b891a559 P ce59d498c5d045519b0d4f5306c4e46d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 11:36:17.763299 23291 tablet_replica.cc:439] T b0dacb6de936447f8c3e49f7b891a559 P f32715fb8fff45afb9e03ef09c7eb305: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5b2c000f19594cfa8696548a481014b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2c000f19594cfa8696548a481014b7" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40535 } } peers { permanent_uuid: "f32715fb8fff45afb9e03ef09c7eb305" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 40175 } } peers { permanent_uuid: "ce59d498c5d045519b0d4f5306c4e46d" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 36341 } } }
I20260806 11:36:17.763422 23291 tablet_replica.cc:442] T b0dacb6de936447f8c3e49f7b891a559 P f32715fb8fff45afb9e03ef09c7eb305: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 11:36:17.764474 23326 mvcc.cc:205] Tried to move back new op lower bound from 7315522264078766080 to 7315522263927640064. Current Snapshot: MvccSnapshot[applied={T|T < 7315522264078766080 or (T in {7315522264078766080})}]
I20260806 11:36:17.772782 23318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:17.768195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33090 (local address 127.21.14.195:36341)
0806 11:36:17.768332 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:17.768338 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:17.768493 (+ 155us) server_negotiation.cc:407] Connection header received
0806 11:36:17.768659 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:17.768661 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:17.768721 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:17.768814 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 11:36:17.769118 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:17.769666 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:17.770957 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:17.771273 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:17.771368 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:17.771681 (+ 313us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 11:36:17.772298 (+ 617us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 11:36:17.772485 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:17.772552 (+ 67us) server_negotiation.cc:299] Negotiation successful
0806 11:36:17.772622 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 11:36:17.786402 23339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:17.782147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.193:40535 (local address 127.21.14.195:36691)
0806 11:36:17.782516 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:17.782530 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:17.782640 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:17.782999 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:17.783004 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:17.783018 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 11:36:17.783188 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:17.783194 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:17.784256 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:17.784259 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:17.784981 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:17.784992 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:17.785649 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:17.785651 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:17.785722 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:17.785781 (+ 59us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 11:36:17.786111 (+ 330us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 11:36:17.786116 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 11:36:17.786168 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 11:36:17.786247 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":135,"threads_started":1}
I20260806 11:36:17.786404 23267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:17.782255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:36691 (local address 127.21.14.193:40535)
0806 11:36:17.782414 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:17.782419 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:17.782686 (+ 267us) server_negotiation.cc:407] Connection header received
0806 11:36:17.782769 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:17.782773 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:17.782834 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:17.782960 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 11:36:17.783326 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:17.784138 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:17.785114 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:17.785482 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:17.785586 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:17.785925 (+ 339us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 11:36:17.786023 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 11:36:17.786100 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:17.786206 (+ 106us) server_negotiation.cc:299] Negotiation successful
0806 11:36:17.786264 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 11:36:17.796671 23050 raft_consensus.cc:1261] T ddb6fd5c5f6e42cdbc4f615042249fbb P f32715fb8fff45afb9e03ef09c7eb305 [term 1 FOLLOWER]: Refusing update from remote peer ce59d498c5d045519b0d4f5306c4e46d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 11:36:17.796695 22902 raft_consensus.cc:1261] T ddb6fd5c5f6e42cdbc4f615042249fbb P 5b2c000f19594cfa8696548a481014b7 [term 1 FOLLOWER]: Refusing update from remote peer ce59d498c5d045519b0d4f5306c4e46d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 11:36:17.797262 23302 consensus_queue.cc:1035] T ddb6fd5c5f6e42cdbc4f615042249fbb P ce59d498c5d045519b0d4f5306c4e46d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f32715fb8fff45afb9e03ef09c7eb305" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 40175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 11:36:17.797525 23276 consensus_queue.cc:1035] T ddb6fd5c5f6e42cdbc4f615042249fbb P ce59d498c5d045519b0d4f5306c4e46d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b2c000f19594cfa8696548a481014b7" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 11:36:17.804633 23324 mvcc.cc:205] Tried to move back new op lower bound from 7315522264249348096 to 7315522262790070272. Current Snapshot: MvccSnapshot[applied={T|T < 7315522264249348096}]
I20260806 11:36:17.829599 23318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:17.825064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:53635 (local address 127.21.14.195:36341)
0806 11:36:17.825220 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:17.825226 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:17.825461 (+ 235us) server_negotiation.cc:407] Connection header received
0806 11:36:17.825596 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:17.825599 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:17.825655 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:17.825745 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 11:36:17.826020 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:17.826890 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:17.828185 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:17.828514 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:17.828599 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:17.829056 (+ 457us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 11:36:17.829150 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 11:36:17.829195 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:17.829354 (+ 159us) server_negotiation.cc:299] Negotiation successful
0806 11:36:17.829432 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 11:36:17.830044 23344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:17.824656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.195:36341 (local address 127.21.14.193:53635)
0806 11:36:17.825367 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:17.825383 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:17.825500 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:17.825781 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:17.825784 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:17.825793 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 11:36:17.825911 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:17.825916 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:17.827042 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:17.827046 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:17.828032 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:17.828046 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:17.828726 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:17.828729 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:17.828813 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:17.828870 (+ 57us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 11:36:17.829259 (+ 389us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 11:36:17.829265 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 11:36:17.829780 (+ 515us) client_negotiation.cc:240] Negotiation successful
0806 11:36:17.829881 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":581,"thread_start_us":130,"threads_started":1}
I20260806 11:36:17.865538 22736 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50350:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 11:36:17.865756 22736 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50350:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 11:36:17.868745 22736 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0569d29f335e4760ad00e3096cef03fc: Sending DeleteTablet for 3 replicas of tablet ddb6fd5c5f6e42cdbc4f615042249fbb
I20260806 11:36:17.869679 22881 tablet_service.cc:1514] Processing DeleteTablet for tablet ddb6fd5c5f6e42cdbc4f615042249fbb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:36:17 UTC) from {username='kudu', principal='kudu/127.21.14.254@KRBTEST.COM'} at 127.0.0.1:46564
I20260806 11:36:17.870018 23029 tablet_service.cc:1514] Processing DeleteTablet for tablet ddb6fd5c5f6e42cdbc4f615042249fbb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:36:17 UTC) from {username='kudu', principal='kudu/127.21.14.254@KRBTEST.COM'} at 127.0.0.1:42554
I20260806 11:36:17.871449 23176 tablet_service.cc:1514] Processing DeleteTablet for tablet ddb6fd5c5f6e42cdbc4f615042249fbb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:36:17 UTC) from {username='kudu', principal='kudu/127.21.14.254@KRBTEST.COM'} at 127.0.0.1:33084
I20260806 11:36:17.871551 23348 tablet_replica.cc:330] T ddb6fd5c5f6e42cdbc4f615042249fbb P f32715fb8fff45afb9e03ef09c7eb305: stopping tablet replica
I20260806 11:36:17.871879 23348 raft_consensus.cc:2229] T ddb6fd5c5f6e42cdbc4f615042249fbb P f32715fb8fff45afb9e03ef09c7eb305 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 11:36:17.872128 23348 raft_consensus.cc:2258] T ddb6fd5c5f6e42cdbc4f615042249fbb P f32715fb8fff45afb9e03ef09c7eb305 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 11:36:17.873255 23348 ts_tablet_manager.cc:1905] T ddb6fd5c5f6e42cdbc4f615042249fbb P f32715fb8fff45afb9e03ef09c7eb305: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 11:36:17.873530 23349 tablet_replica.cc:330] T ddb6fd5c5f6e42cdbc4f615042249fbb P ce59d498c5d045519b0d4f5306c4e46d: stopping tablet replica
I20260806 11:36:17.873682 23349 raft_consensus.cc:2229] T ddb6fd5c5f6e42cdbc4f615042249fbb P ce59d498c5d045519b0d4f5306c4e46d [term 1 LEADER]: Raft consensus shutting down.
I20260806 11:36:17.874040 23349 raft_consensus.cc:2258] T ddb6fd5c5f6e42cdbc4f615042249fbb P ce59d498c5d045519b0d4f5306c4e46d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 11:36:17.874878 23349 ts_tablet_manager.cc:1905] T ddb6fd5c5f6e42cdbc4f615042249fbb P ce59d498c5d045519b0d4f5306c4e46d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 11:36:17.877271 23348 ts_tablet_manager.cc:1918] T ddb6fd5c5f6e42cdbc4f615042249fbb P f32715fb8fff45afb9e03ef09c7eb305: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 11:36:17.877461 23348 log.cc:1196] T ddb6fd5c5f6e42cdbc4f615042249fbb P f32715fb8fff45afb9e03ef09c7eb305: Deleting WAL directory at /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/ts-1/wal/wals/ddb6fd5c5f6e42cdbc4f615042249fbb
I20260806 11:36:17.877902 23348 ts_tablet_manager.cc:1939] T ddb6fd5c5f6e42cdbc4f615042249fbb P f32715fb8fff45afb9e03ef09c7eb305: Deleting consensus metadata
I20260806 11:36:17.878165 23349 ts_tablet_manager.cc:1918] T ddb6fd5c5f6e42cdbc4f615042249fbb P ce59d498c5d045519b0d4f5306c4e46d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 11:36:17.878264 23349 log.cc:1196] T ddb6fd5c5f6e42cdbc4f615042249fbb P ce59d498c5d045519b0d4f5306c4e46d: Deleting WAL directory at /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/ts-2/wal/wals/ddb6fd5c5f6e42cdbc4f615042249fbb
I20260806 11:36:17.878558 23349 ts_tablet_manager.cc:1939] T ddb6fd5c5f6e42cdbc4f615042249fbb P ce59d498c5d045519b0d4f5306c4e46d: Deleting consensus metadata
I20260806 11:36:17.878558 23347 tablet_replica.cc:330] T ddb6fd5c5f6e42cdbc4f615042249fbb P 5b2c000f19594cfa8696548a481014b7: stopping tablet replica
I20260806 11:36:17.879562 22720 catalog_manager.cc:4841] TS ce59d498c5d045519b0d4f5306c4e46d (127.21.14.195:36341): tablet ddb6fd5c5f6e42cdbc4f615042249fbb (table test-table [id=e72288dc83e74235b2a4b709efceda40]) successfully deleted
I20260806 11:36:17.879562 22722 catalog_manager.cc:4841] TS f32715fb8fff45afb9e03ef09c7eb305 (127.21.14.194:40175): tablet ddb6fd5c5f6e42cdbc4f615042249fbb (table test-table [id=e72288dc83e74235b2a4b709efceda40]) successfully deleted
I20260806 11:36:17.879879 23347 raft_consensus.cc:2229] T ddb6fd5c5f6e42cdbc4f615042249fbb P 5b2c000f19594cfa8696548a481014b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 11:36:17.880115 23347 raft_consensus.cc:2258] T ddb6fd5c5f6e42cdbc4f615042249fbb P 5b2c000f19594cfa8696548a481014b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 11:36:17.881295 23347 ts_tablet_manager.cc:1905] T ddb6fd5c5f6e42cdbc4f615042249fbb P 5b2c000f19594cfa8696548a481014b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 11:36:17.884907 23347 ts_tablet_manager.cc:1918] T ddb6fd5c5f6e42cdbc4f615042249fbb P 5b2c000f19594cfa8696548a481014b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 11:36:17.885023 23347 log.cc:1196] T ddb6fd5c5f6e42cdbc4f615042249fbb P 5b2c000f19594cfa8696548a481014b7: Deleting WAL directory at /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786016169045816-21563-0/minicluster-data/ts-0/wal/wals/ddb6fd5c5f6e42cdbc4f615042249fbb
I20260806 11:36:17.885465 23347 ts_tablet_manager.cc:1939] T ddb6fd5c5f6e42cdbc4f615042249fbb P 5b2c000f19594cfa8696548a481014b7: Deleting consensus metadata
I20260806 11:36:17.886497 22724 catalog_manager.cc:4841] TS 5b2c000f19594cfa8696548a481014b7 (127.21.14.193:40535): tablet ddb6fd5c5f6e42cdbc4f615042249fbb (table test-table [id=e72288dc83e74235b2a4b709efceda40]) successfully deleted
Aug 06 11:36:17 dist-test-slave-7c35 krb5kdc[22690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016176, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.14.193@KRBTEST.COM
I20260806 11:36:17.900156 23267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:17.880357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46584 (local address 127.21.14.193:40535)
0806 11:36:17.880634 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:17.880639 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:17.886665 (+ 6026us) server_negotiation.cc:407] Connection header received
0806 11:36:17.886951 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:17.886954 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:17.887011 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:17.887107 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:17.888098 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:17.888921 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:17.890166 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:17.890534 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:17.890646 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:17.895315 (+ 4669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:17.895327 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:17.895346 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:17.895390 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:17.897737 (+ 2347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:17.898388 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:17.898399 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:17.898401 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:17.898554 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:17.898998 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:17.899002 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:17.899004 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:17.899363 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:17.899668 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:17.899820 (+ 152us) server_negotiation.cc:299] Negotiation successful
0806 11:36:17.899968 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187}
W20260806 11:36:17.900978 22944 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46584
I20260806 11:36:17.915148 23311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:17.903155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50360 (local address 127.21.14.254:40925)
0806 11:36:17.903320 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:17.903325 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:17.903550 (+ 225us) server_negotiation.cc:407] Connection header received
0806 11:36:17.903800 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:17.903803 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:17.903858 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:17.903913 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:17.904932 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:17.905798 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:17.907104 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:17.907504 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:17.907589 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:17.910923 (+ 3334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:17.910928 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:17.910931 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:17.910972 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:17.912883 (+ 1911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:17.913536 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:17.913541 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:17.913543 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:17.913650 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:17.913984 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:17.913987 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:17.913989 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:17.914239 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:17.914350 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:17.914749 (+ 399us) server_negotiation.cc:299] Negotiation successful
0806 11:36:17.914958 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
W20260806 11:36:17.916154 22736 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50360
I20260806 11:36:17.917682 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 22813
I20260806 11:36:17.930954 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 22960
I20260806 11:36:17.941258 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 23108
I20260806 11:36:17.950521 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 22703
2026-08-06T11:36:17Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (3461 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 11:36:18 dist-test-slave-7c35 krb5kdc[23356](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 11:36:18 dist-test-slave-7c35 krb5kdc[23356](info): set up 2 sockets
Aug 06 11:36:18 dist-test-slave-7c35 krb5kdc[23356](info): commencing operation
krb5kdc: starting...
W20260806 11:36:19.034329 21563 mini_kdc.cc:121] Time spent starting KDC: real 1.057s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Aug 06 11:36:19 dist-test-slave-7c35 krb5kdc[23356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016179, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T11:36:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T11:36:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.14.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab.
Entry for principal kudu/127.21.14.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab.
I20260806 11:36:19.202253 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:40863
--webserver_interface=127.21.14.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab
--principal=kudu/127.21.14.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:34477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.14.254:40863
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:36:19.337379 23370 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:36:19.337780 23370 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:36:19.337872 23370 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:36:19.342257 23370 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 11:36:19.342370 23370 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 11:36:19.342401 23370 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:36:19.342442 23370 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:36:19.342497 23370 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 11:36:19.342537 23370 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 11:36:19.348692 23370 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:34477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.14.254:40863
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab
--principal=kudu/127.21.14.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:40863
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.14.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23370
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:36:19.350339 23370 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:36:19.351644 23370 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 11:36:19.362481 23370 server_base.cc:1034] running on GCE node
W20260806 11:36:19.363698 23376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:19.363703 23378 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:19.363907 23375 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:19.364663 23370 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:36:19.366042 23370 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:36:19.367234 23370 hybrid_clock.cc:648] HybridClock initialized: now 1786016179367197 us; error 58 us; skew 500 ppm
Aug 06 11:36:19 dist-test-slave-7c35 krb5kdc[23356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016179, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:36:19.370456 23370 init.cc:377] Logged in from keytab as kudu/127.21.14.254@KRBTEST.COM (short username kudu)
I20260806 11:36:19.371692 23370 webserver.cc:460] Webserver started at http://127.21.14.254:41433/ using document root <none> and password file <none>
I20260806 11:36:19.372310 23370 fs_manager.cc:359] Metadata directory not provided
I20260806 11:36:19.372377 23370 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:36:19.372592 23370 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:36:19.375170 23370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/master-0/data/instance:
uuid: "de08b384a69445a79d55b1cef258a14e"
format_stamp: "Formatted at 2026-08-06 11:36:19 on dist-test-slave-7c35"
server_key: "dabedfed4afd6684872aaa9ef1dfc0f0"
server_key_iv: "5cc2d25ed166363c2789e9cd9421b613"
server_key_version: "encryptionkey@0"
I20260806 11:36:19.375833 23370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/master-0/wal/instance:
uuid: "de08b384a69445a79d55b1cef258a14e"
format_stamp: "Formatted at 2026-08-06 11:36:19 on dist-test-slave-7c35"
server_key: "dabedfed4afd6684872aaa9ef1dfc0f0"
server_key_iv: "5cc2d25ed166363c2789e9cd9421b613"
server_key_version: "encryptionkey@0"
I20260806 11:36:19.380615 23370 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 11:36:19.383572 23385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:19.385115 23370 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 11:36:19.385247 23370 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/master-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/master-0/wal
uuid: "de08b384a69445a79d55b1cef258a14e"
format_stamp: "Formatted at 2026-08-06 11:36:19 on dist-test-slave-7c35"
server_key: "dabedfed4afd6684872aaa9ef1dfc0f0"
server_key_iv: "5cc2d25ed166363c2789e9cd9421b613"
server_key_version: "encryptionkey@0"
I20260806 11:36:19.385363 23370 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:36:19.403820 23370 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:36:19.407603 23370 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:36:19.407793 23370 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:36:19.417753 23370 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.254:40863
I20260806 11:36:19.417873 23453 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.254:40863 every 8 connection(s)
I20260806 11:36:19.419101 23370 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
I20260806 11:36:19.421540 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 23370
I20260806 11:36:19.421684 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/master-0/wal/instance
I20260806 11:36:19.422005 21563 external_mini_cluster.cc:1428] Setting key f094f5c760d74caead0080b4dbf5eada
I20260806 11:36:19.423219 23454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:19.431591 23454 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de08b384a69445a79d55b1cef258a14e: Bootstrap starting.
Aug 06 11:36:19 dist-test-slave-7c35 krb5kdc[23356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016179, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:36:19.435210 23454 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P de08b384a69445a79d55b1cef258a14e: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:19.436908 23454 log.cc:824] T 00000000000000000000000000000000 P de08b384a69445a79d55b1cef258a14e: Log is configured to *not* fsync() on all Append() calls
I20260806 11:36:19.439442 23454 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de08b384a69445a79d55b1cef258a14e: No bootstrap required, opened a new log
I20260806 11:36:19.440798 23457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:19.423795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39138 (local address 127.21.14.254:40863)
0806 11:36:19.424314 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:19.424332 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:19.424376 (+ 44us) server_negotiation.cc:407] Connection header received
0806 11:36:19.425401 (+ 1025us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:19.425431 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:19.425915 (+ 484us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:19.426302 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:19.427547 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:19.428720 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:19.430068 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:19.430659 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:19.430807 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:19.434836 (+ 4029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:19.434843 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:19.434934 (+ 91us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:19.434979 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:19.437345 (+ 2366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:19.437942 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:19.437948 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:19.437954 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:19.438097 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:19.438437 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:19.438446 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:19.438448 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:19.438988 (+ 540us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:19.439236 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:19.439574 (+ 338us) server_negotiation.cc:299] Negotiation successful
0806 11:36:19.439935 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":176,"threads_started":1}
I20260806 11:36:19.443490 23454 raft_consensus.cc:348] T 00000000000000000000000000000000 P de08b384a69445a79d55b1cef258a14e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de08b384a69445a79d55b1cef258a14e" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 40863 } }
I20260806 11:36:19.443907 23454 raft_consensus.cc:374] T 00000000000000000000000000000000 P de08b384a69445a79d55b1cef258a14e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:19.443998 23454 raft_consensus.cc:729] T 00000000000000000000000000000000 P de08b384a69445a79d55b1cef258a14e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de08b384a69445a79d55b1cef258a14e, State: Initialized, Role: FOLLOWER
I20260806 11:36:19.444479 23454 consensus_queue.cc:260] T 00000000000000000000000000000000 P de08b384a69445a79d55b1cef258a14e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de08b384a69445a79d55b1cef258a14e" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 40863 } }
I20260806 11:36:19.444658 23454 raft_consensus.cc:388] T 00000000000000000000000000000000 P de08b384a69445a79d55b1cef258a14e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 11:36:19.444746 23454 raft_consensus.cc:482] T 00000000000000000000000000000000 P de08b384a69445a79d55b1cef258a14e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 11:36:19.444859 23454 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de08b384a69445a79d55b1cef258a14e [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:36:19.446046 23454 raft_consensus.cc:504] T 00000000000000000000000000000000 P de08b384a69445a79d55b1cef258a14e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de08b384a69445a79d55b1cef258a14e" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 40863 } }
I20260806 11:36:19.446466 23454 leader_election.cc:302] T 00000000000000000000000000000000 P de08b384a69445a79d55b1cef258a14e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: de08b384a69445a79d55b1cef258a14e; no voters:
I20260806 11:36:19.446950 23454 leader_election.cc:288] T 00000000000000000000000000000000 P de08b384a69445a79d55b1cef258a14e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 11:36:19.447199 23459 raft_consensus.cc:2781] T 00000000000000000000000000000000 P de08b384a69445a79d55b1cef258a14e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:36:19.447496 23459 raft_consensus.cc:686] T 00000000000000000000000000000000 P de08b384a69445a79d55b1cef258a14e [term 1 LEADER]: Becoming Leader. State: Replica: de08b384a69445a79d55b1cef258a14e, State: Running, Role: LEADER
I20260806 11:36:19.447953 23459 consensus_queue.cc:237] T 00000000000000000000000000000000 P de08b384a69445a79d55b1cef258a14e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de08b384a69445a79d55b1cef258a14e" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 40863 } }
I20260806 11:36:19.448963 23454 sys_catalog.cc:564] T 00000000000000000000000000000000 P de08b384a69445a79d55b1cef258a14e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 11:36:19.450210 23461 sys_catalog.cc:455] T 00000000000000000000000000000000 P de08b384a69445a79d55b1cef258a14e [sys.catalog]: SysCatalogTable state changed. Reason: New leader de08b384a69445a79d55b1cef258a14e. Latest consensus state: current_term: 1 leader_uuid: "de08b384a69445a79d55b1cef258a14e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de08b384a69445a79d55b1cef258a14e" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 40863 } } }
I20260806 11:36:19.450421 23461 sys_catalog.cc:458] T 00000000000000000000000000000000 P de08b384a69445a79d55b1cef258a14e [sys.catalog]: This master's current role is: LEADER
I20260806 11:36:19.450768 23460 sys_catalog.cc:455] T 00000000000000000000000000000000 P de08b384a69445a79d55b1cef258a14e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "de08b384a69445a79d55b1cef258a14e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de08b384a69445a79d55b1cef258a14e" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 40863 } } }
I20260806 11:36:19.450897 23460 sys_catalog.cc:458] T 00000000000000000000000000000000 P de08b384a69445a79d55b1cef258a14e [sys.catalog]: This master's current role is: LEADER
I20260806 11:36:19.450892 23468 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 11:36:19.455008 23468 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 11:36:19.462965 23468 catalog_manager.cc:1347] Generated new cluster ID: a11efc4b109a419ab01e2ada6d730bcf
I20260806 11:36:19.463088 23468 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 11:36:19.516666 23468 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 11:36:19.517803 23468 catalog_manager.cc:1504] Loading token signing keys...
I20260806 11:36:19.528048 23468 catalog_manager.cc:5868] T 00000000000000000000000000000000 P de08b384a69445a79d55b1cef258a14e: Generated new TSK 0
I20260806 11:36:19.529036 23468 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.14.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab.
Entry for principal kudu/127.21.14.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab.
I20260806 11:36:19.566767 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.193:0
--local_ip_for_outbound_sockets=127.21.14.193
--webserver_interface=127.21.14.193
--webserver_port=0
--tserver_master_addrs=127.21.14.254:40863
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab
--principal=kudu/127.21.14.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:34477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:36:19.700690 23481 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:36:19.701076 23481 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:36:19.701165 23481 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:36:19.705842 23481 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:36:19.705984 23481 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:36:19.706068 23481 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 11:36:19.706111 23481 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 11:36:19.706174 23481 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.193
I20260806 11:36:19.712252 23481 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:34477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab
--principal=kudu/127.21.14.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.14.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.14.254:40863
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23481
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.193
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:36:19.713523 23481 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:36:19.714643 23481 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:36:19.725495 23488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:19.725498 23487 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:19.725502 23490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:19.725620 23481 server_base.cc:1034] running on GCE node
I20260806 11:36:19.726254 23481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:36:19.727015 23481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:36:19.728210 23481 hybrid_clock.cc:648] HybridClock initialized: now 1786016179728158 us; error 64 us; skew 500 ppm
Aug 06 11:36:19 dist-test-slave-7c35 krb5kdc[23356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016179, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:36:19.731326 23481 init.cc:377] Logged in from keytab as kudu/127.21.14.193@KRBTEST.COM (short username kudu)
I20260806 11:36:19.732549 23481 webserver.cc:460] Webserver started at http://127.21.14.193:43133/ using document root <none> and password file <none>
I20260806 11:36:19.733239 23481 fs_manager.cc:359] Metadata directory not provided
I20260806 11:36:19.733349 23481 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:36:19.733615 23481 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:36:19.736064 23481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/ts-0/data/instance:
uuid: "c22a8b491fc74a76a11e016052b574ab"
format_stamp: "Formatted at 2026-08-06 11:36:19 on dist-test-slave-7c35"
server_key: "6594ca3c5802a665aba8f4c5e1873b47"
server_key_iv: "b29849d15c219799da2e12fc9ec7fd69"
server_key_version: "encryptionkey@0"
I20260806 11:36:19.736742 23481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/ts-0/wal/instance:
uuid: "c22a8b491fc74a76a11e016052b574ab"
format_stamp: "Formatted at 2026-08-06 11:36:19 on dist-test-slave-7c35"
server_key: "6594ca3c5802a665aba8f4c5e1873b47"
server_key_iv: "b29849d15c219799da2e12fc9ec7fd69"
server_key_version: "encryptionkey@0"
I20260806 11:36:19.741145 23481 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 11:36:19.744112 23497 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:19.745623 23481 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 11:36:19.745815 23481 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/ts-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/ts-0/wal
uuid: "c22a8b491fc74a76a11e016052b574ab"
format_stamp: "Formatted at 2026-08-06 11:36:19 on dist-test-slave-7c35"
server_key: "6594ca3c5802a665aba8f4c5e1873b47"
server_key_iv: "b29849d15c219799da2e12fc9ec7fd69"
server_key_version: "encryptionkey@0"
I20260806 11:36:19.745968 23481 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:36:19.772832 23481 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:36:19.776539 23481 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:36:19.776767 23481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:36:19.778460 23481 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:36:19.778541 23481 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:19.778618 23481 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:36:19.778658 23481 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 11:36:19 dist-test-slave-7c35 krb5kdc[23356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016179, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.193@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:36:19.790576 23481 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.193:42373
I20260806 11:36:19.790691 23617 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.193:42373 every 8 connection(s)
I20260806 11:36:19.791858 23481 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
I20260806 11:36:19.793661 23457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:19.779460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:55271 (local address 127.21.14.254:40863)
0806 11:36:19.779597 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:19.779601 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:19.780841 (+ 1240us) server_negotiation.cc:407] Connection header received
0806 11:36:19.782011 (+ 1170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:19.782015 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:19.782080 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:19.782186 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:19.783696 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:19.784282 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:19.785608 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:19.785857 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:19.785975 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:19.789323 (+ 3348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:19.789327 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:19.789329 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:19.789358 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:19.791309 (+ 1951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:19.791912 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:19.791916 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:19.791918 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:19.792025 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:19.792398 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:19.792402 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:19.792406 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:19.792693 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:19.792805 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:19.793326 (+ 521us) server_negotiation.cc:299] Negotiation successful
0806 11:36:19.793482 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 11:36:19.794524 23506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:19.780094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:40863 (local address 127.21.14.193:55271)
0806 11:36:19.780655 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:19.780703 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:19.781761 (+ 1058us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:19.782414 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:19.782421 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:19.782792 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:19.783516 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:19.783539 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:19.784424 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:19.784427 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:19.785470 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:19.785481 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:19.786039 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:19.786043 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:19.786255 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:19.787036 (+ 781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:19.787064 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:19.789100 (+ 2036us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:19.791451 (+ 2351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:19.791457 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:19.791470 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:19.791786 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:19.792127 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:19.792130 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:19.792131 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:19.792291 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:19.792801 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:19.792808 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:19.793072 (+ 264us) client_negotiation.cc:769] Sending connection context
0806 11:36:19.793274 (+ 202us) client_negotiation.cc:240] Negotiation successful
0806 11:36:19.793643 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":142,"threads_started":1}
I20260806 11:36:19.796089 23618 heartbeater.cc:347] Connected to a master server at 127.21.14.254:40863
I20260806 11:36:19.796273 23618 heartbeater.cc:464] Registering TS with master...
I20260806 11:36:19.796931 23618 heartbeater.cc:511] Master 127.21.14.254:40863 requested a full tablet report, sending...
I20260806 11:36:19.798808 23401 ts_manager.cc:194] Registered new tserver with Master: c22a8b491fc74a76a11e016052b574ab (127.21.14.193:42373)
I20260806 11:36:19.800209 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 23481
I20260806 11:36:19.800346 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/ts-0/wal/instance
I20260806 11:36:19.800252 23401 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.14.193@KRBTEST.COM'} at 127.21.14.193:55271
I20260806 11:36:19.800717 21563 external_mini_cluster.cc:1428] Setting key 4fbee01672288c4f8182deefcbad116d
I20260806 11:36:19.807407 23457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:19.798416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:47977 (local address 127.21.14.254:40863)
0806 11:36:19.798561 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:19.798565 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:19.798902 (+ 337us) server_negotiation.cc:407] Connection header received
0806 11:36:19.799026 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:19.799030 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:19.799082 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:19.799165 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:19.799724 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:19.800489 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:19.801565 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:19.801836 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:19.801959 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:19.803359 (+ 1400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:19.803365 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:19.803371 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:19.803406 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:19.805383 (+ 1977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:19.806030 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:19.806034 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:19.806035 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:19.806125 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:19.806470 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:19.806473 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:19.806474 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:19.806673 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:19.806751 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:19.807152 (+ 401us) server_negotiation.cc:299] Negotiation successful
0806 11:36:19.807276 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 11:36:19.807583 23625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:19.798298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:40863 (local address 127.21.14.193:47977)
0806 11:36:19.798737 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:19.798752 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:19.798910 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:19.799210 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:19.799213 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:19.799453 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:19.799596 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:19.799604 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:19.800656 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:19.800660 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:19.801442 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:19.801454 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:19.801970 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:19.801974 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:19.802083 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:19.802669 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:19.802685 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:19.803202 (+ 517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:19.805543 (+ 2341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:19.805548 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:19.805552 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:19.805892 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:19.806246 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:19.806250 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:19.806254 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:19.806366 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:19.806771 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:19.806777 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:19.806965 (+ 188us) client_negotiation.cc:769] Sending connection context
0806 11:36:19.807090 (+ 125us) client_negotiation.cc:240] Negotiation successful
0806 11:36:19.807290 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":107,"threads_started":1}
WARNING: no policy specified for kudu/127.21.14.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab.
Entry for principal kudu/127.21.14.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab.
I20260806 11:36:19.835407 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.194:0
--local_ip_for_outbound_sockets=127.21.14.194
--webserver_interface=127.21.14.194
--webserver_port=0
--tserver_master_addrs=127.21.14.254:40863
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab
--principal=kudu/127.21.14.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:34477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:36:19.960773 23628 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:36:19.961086 23628 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:36:19.961138 23628 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:36:19.965809 23628 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:36:19.965920 23628 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:36:19.965967 23628 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 11:36:19.965981 23628 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 11:36:19.966017 23628 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.194
I20260806 11:36:19.972496 23628 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:34477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab
--principal=kudu/127.21.14.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.14.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.14.254:40863
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23628
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.194
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:36:19.973724 23628 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:36:19.974823 23628 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:36:19.985739 23636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:19.985963 23633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:19.985771 23634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:19.986220 23628 server_base.cc:1034] running on GCE node
I20260806 11:36:19.986739 23628 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:36:19.987548 23628 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:36:19.988731 23628 hybrid_clock.cc:648] HybridClock initialized: now 1786016179988691 us; error 56 us; skew 500 ppm
Aug 06 11:36:19 dist-test-slave-7c35 krb5kdc[23356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016179, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:36:19.992029 23628 init.cc:377] Logged in from keytab as kudu/127.21.14.194@KRBTEST.COM (short username kudu)
I20260806 11:36:19.993197 23628 webserver.cc:460] Webserver started at http://127.21.14.194:33547/ using document root <none> and password file <none>
I20260806 11:36:19.993860 23628 fs_manager.cc:359] Metadata directory not provided
I20260806 11:36:19.993965 23628 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:36:19.994232 23628 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:36:19.996721 23628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/ts-1/data/instance:
uuid: "f62920d5c6274e8987acdec21c17ab85"
format_stamp: "Formatted at 2026-08-06 11:36:19 on dist-test-slave-7c35"
server_key: "da684619457716096c8eaf54164c7220"
server_key_iv: "f1729aa57117b895cae7efc67c23ce76"
server_key_version: "encryptionkey@0"
I20260806 11:36:19.997375 23628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/ts-1/wal/instance:
uuid: "f62920d5c6274e8987acdec21c17ab85"
format_stamp: "Formatted at 2026-08-06 11:36:19 on dist-test-slave-7c35"
server_key: "da684619457716096c8eaf54164c7220"
server_key_iv: "f1729aa57117b895cae7efc67c23ce76"
server_key_version: "encryptionkey@0"
I20260806 11:36:20.001706 23628 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 11:36:20.004669 23644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:20.006197 23628 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 11:36:20.006366 23628 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/ts-1/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/ts-1/wal
uuid: "f62920d5c6274e8987acdec21c17ab85"
format_stamp: "Formatted at 2026-08-06 11:36:19 on dist-test-slave-7c35"
server_key: "da684619457716096c8eaf54164c7220"
server_key_iv: "f1729aa57117b895cae7efc67c23ce76"
server_key_version: "encryptionkey@0"
I20260806 11:36:20.006515 23628 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:36:20.031858 23628 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:36:20.035563 23628 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:36:20.035796 23628 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:36:20.037395 23628 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:36:20.037498 23628 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:20.037576 23628 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:36:20.037616 23628 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 11:36:20 dist-test-slave-7c35 krb5kdc[23356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016179, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.194@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:36:20.050405 23628 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.194:33067
I20260806 11:36:20.050441 23761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.194:33067 every 8 connection(s)
I20260806 11:36:20.052047 23628 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
I20260806 11:36:20.054123 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 23628
I20260806 11:36:20.054363 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/ts-1/wal/instance
I20260806 11:36:20.054742 21563 external_mini_cluster.cc:1428] Setting key f0426c336f5d3c2346a4857e3c66580a
I20260806 11:36:20.055015 23457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:20.038532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:33197 (local address 127.21.14.254:40863)
0806 11:36:20.038670 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:20.038673 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:20.040034 (+ 1361us) server_negotiation.cc:407] Connection header received
0806 11:36:20.041308 (+ 1274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:20.041315 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:20.041396 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:20.041494 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:20.043411 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.044342 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.045949 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.046287 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.046415 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:20.049847 (+ 3432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:20.049861 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:20.049865 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:20.049931 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:20.052141 (+ 2210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:20.052901 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:20.052905 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:20.052907 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:20.053053 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:20.053493 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:20.053497 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:20.053498 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:20.053845 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:20.053964 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:20.054627 (+ 663us) server_negotiation.cc:299] Negotiation successful
0806 11:36:20.054807 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 11:36:20.056025 23653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:20.039235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:40863 (local address 127.21.14.194:33197)
0806 11:36:20.039842 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:20.039893 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:20.040989 (+ 1096us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:20.041756 (+ 767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:20.041770 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:20.042339 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:20.043203 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:20.043225 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.044579 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.044583 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:20.045786 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:20.045799 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.046473 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.046478 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:20.046696 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:20.047611 (+ 915us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:20.047637 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:20.049558 (+ 1921us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:20.052329 (+ 2771us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:20.052336 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:20.052353 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:20.052738 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:20.053186 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:20.053189 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:20.053191 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:20.053375 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:20.053983 (+ 608us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:20.053995 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:20.054284 (+ 289us) client_negotiation.cc:769] Sending connection context
0806 11:36:20.054514 (+ 230us) client_negotiation.cc:240] Negotiation successful
0806 11:36:20.054899 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":144,"threads_started":1}
I20260806 11:36:20.057487 23762 heartbeater.cc:347] Connected to a master server at 127.21.14.254:40863
I20260806 11:36:20.057693 23762 heartbeater.cc:464] Registering TS with master...
I20260806 11:36:20.058462 23762 heartbeater.cc:511] Master 127.21.14.254:40863 requested a full tablet report, sending...
I20260806 11:36:20.060017 23401 ts_manager.cc:194] Registered new tserver with Master: f62920d5c6274e8987acdec21c17ab85 (127.21.14.194:33067)
I20260806 11:36:20.060786 23401 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.14.194@KRBTEST.COM'} at 127.21.14.194:33197
WARNING: no policy specified for kudu/127.21.14.195@KRBTEST.COM; defaulting to no policy
I20260806 11:36:20.069018 23769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:20.060151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:40863 (local address 127.21.14.194:34937)
0806 11:36:20.060601 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:20.060614 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:20.060741 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:20.061022 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:20.061025 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:20.061261 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:20.061373 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:20.061379 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.062169 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.062173 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:20.062794 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:20.062808 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.063504 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.063507 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:20.063592 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:20.064283 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:20.064297 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:20.064647 (+ 350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:20.067083 (+ 2436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:20.067086 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:20.067089 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:20.067373 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:20.067732 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:20.067735 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:20.067736 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:20.067820 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:20.068508 (+ 688us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:20.068510 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:20.068604 (+ 94us) client_negotiation.cc:769] Sending connection context
0806 11:36:20.068705 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 11:36:20.068837 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":136,"threads_started":1}
I20260806 11:36:20.069026 23457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:20.060234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:34937 (local address 127.21.14.254:40863)
0806 11:36:20.060362 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:20.060366 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:20.060692 (+ 326us) server_negotiation.cc:407] Connection header received
0806 11:36:20.060851 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:20.060854 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:20.060902 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:20.060995 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:20.061496 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.062033 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.062958 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.063360 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.063493 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:20.064823 (+ 1330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:20.064829 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:20.064832 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:20.064868 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:20.066933 (+ 2065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:20.067495 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:20.067501 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:20.067506 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:20.067629 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:20.067909 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:20.067913 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:20.067916 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:20.068366 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:20.068527 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:20.068778 (+ 251us) server_negotiation.cc:299] Negotiation successful
0806 11:36:20.068902 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab.
Entry for principal kudu/127.21.14.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab.
I20260806 11:36:20.091542 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.195:0
--local_ip_for_outbound_sockets=127.21.14.195
--webserver_interface=127.21.14.195
--webserver_port=0
--tserver_master_addrs=127.21.14.254:40863
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab
--principal=kudu/127.21.14.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:34477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:36:20.229866 23771 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:36:20.230265 23771 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:36:20.230368 23771 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:36:20.235814 23771 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:36:20.235972 23771 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:36:20.236056 23771 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 11:36:20.236169 23771 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 11:36:20.236320 23771 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.195
I20260806 11:36:20.243158 23771 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:34477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab
--principal=kudu/127.21.14.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.14.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.14.254:40863
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23771
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.195
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:36:20.244505 23771 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:36:20.245716 23771 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:36:20.258566 23779 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:20.258540 23781 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:20.258831 23771 server_base.cc:1034] running on GCE node
W20260806 11:36:20.258560 23778 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:20.259369 23771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:36:20.260154 23771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:36:20.261353 23771 hybrid_clock.cc:648] HybridClock initialized: now 1786016180261339 us; error 30 us; skew 500 ppm
Aug 06 11:36:20 dist-test-slave-7c35 krb5kdc[23356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016180, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:36:20.264632 23771 init.cc:377] Logged in from keytab as kudu/127.21.14.195@KRBTEST.COM (short username kudu)
I20260806 11:36:20.265692 23771 webserver.cc:460] Webserver started at http://127.21.14.195:43653/ using document root <none> and password file <none>
I20260806 11:36:20.266321 23771 fs_manager.cc:359] Metadata directory not provided
I20260806 11:36:20.266415 23771 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:36:20.266660 23771 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:36:20.269210 23771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/ts-2/data/instance:
uuid: "66ca4b1903f6452fa22ce688b579d7eb"
format_stamp: "Formatted at 2026-08-06 11:36:20 on dist-test-slave-7c35"
server_key: "efbe7f8d67fc859cd796d40252955adf"
server_key_iv: "fc5e767be0f2df84f721d48d1089b725"
server_key_version: "encryptionkey@0"
I20260806 11:36:20.269935 23771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/ts-2/wal/instance:
uuid: "66ca4b1903f6452fa22ce688b579d7eb"
format_stamp: "Formatted at 2026-08-06 11:36:20 on dist-test-slave-7c35"
server_key: "efbe7f8d67fc859cd796d40252955adf"
server_key_iv: "fc5e767be0f2df84f721d48d1089b725"
server_key_version: "encryptionkey@0"
I20260806 11:36:20.274349 23771 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.002s
I20260806 11:36:20.277379 23788 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:20.278990 23771 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 11:36:20.279148 23771 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/ts-2/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/ts-2/wal
uuid: "66ca4b1903f6452fa22ce688b579d7eb"
format_stamp: "Formatted at 2026-08-06 11:36:20 on dist-test-slave-7c35"
server_key: "efbe7f8d67fc859cd796d40252955adf"
server_key_iv: "fc5e767be0f2df84f721d48d1089b725"
server_key_version: "encryptionkey@0"
I20260806 11:36:20.279300 23771 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:36:20.299240 23771 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:36:20.303346 23771 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:36:20.303591 23771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:36:20.305236 23771 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:36:20.305320 23771 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:20.305389 23771 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:36:20.305428 23771 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 11:36:20 dist-test-slave-7c35 krb5kdc[23356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016180, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.195@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:36:20.317503 23771 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.195:43505
I20260806 11:36:20.318885 23771 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
I20260806 11:36:20.319028 23905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.195:43505 every 8 connection(s)
I20260806 11:36:20.321251 23457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:20.306354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:50873 (local address 127.21.14.254:40863)
0806 11:36:20.306520 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:20.306525 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:20.307787 (+ 1262us) server_negotiation.cc:407] Connection header received
0806 11:36:20.309166 (+ 1379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:20.309170 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:20.309239 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:20.309350 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:20.311060 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.311914 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.313336 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.313580 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.313694 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:20.316783 (+ 3089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:20.316787 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:20.316794 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:20.316825 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:20.318635 (+ 1810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:20.319492 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:20.319496 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:20.319497 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:20.319603 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:20.320031 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:20.320035 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:20.320036 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:20.320271 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:20.320360 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:20.320995 (+ 635us) server_negotiation.cc:299] Negotiation successful
0806 11:36:20.321121 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 11:36:20.322083 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 23771
I20260806 11:36:20.322225 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/ts-2/wal/instance
I20260806 11:36:20.322393 23797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:20.306953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:40863 (local address 127.21.14.195:50873)
0806 11:36:20.307586 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:20.307644 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:20.308923 (+ 1279us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:20.309569 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:20.309579 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:20.310138 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:20.310840 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:20.310899 (+ 59us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.312070 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.312074 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:20.313200 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:20.313212 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.313767 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.313772 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:20.313961 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:20.314667 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:20.314685 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:20.316573 (+ 1888us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:20.318790 (+ 2217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:20.318796 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:20.318811 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:20.319312 (+ 501us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:20.319720 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:20.319723 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:20.319725 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:20.319910 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:20.320421 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:20.320427 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:20.320703 (+ 276us) client_negotiation.cc:769] Sending connection context
0806 11:36:20.320891 (+ 188us) client_negotiation.cc:240] Negotiation successful
0806 11:36:20.321222 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":156,"threads_started":1}
I20260806 11:36:20.322562 21563 external_mini_cluster.cc:1428] Setting key c59455a74dd6afb6fdbcfe2878bf70f5
I20260806 11:36:20.323933 23906 heartbeater.cc:347] Connected to a master server at 127.21.14.254:40863
I20260806 11:36:20.324877 23906 heartbeater.cc:464] Registering TS with master...
I20260806 11:36:20.325719 23906 heartbeater.cc:511] Master 127.21.14.254:40863 requested a full tablet report, sending...
I20260806 11:36:20.327066 23402 ts_manager.cc:194] Registered new tserver with Master: 66ca4b1903f6452fa22ce688b579d7eb (127.21.14.195:43505)
I20260806 11:36:20.327733 23402 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.14.195@KRBTEST.COM'} at 127.21.14.195:50873
I20260806 11:36:20.334142 23457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:20.326838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:59907 (local address 127.21.14.254:40863)
0806 11:36:20.327012 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:20.327019 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:20.327115 (+ 96us) server_negotiation.cc:407] Connection header received
0806 11:36:20.327323 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:20.327326 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:20.327377 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:20.327458 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:20.327970 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.328650 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.329488 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.329726 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.329830 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:20.331138 (+ 1308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:20.331141 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:20.331144 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:20.331177 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:20.332406 (+ 1229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:20.332849 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:20.332853 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:20.332854 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:20.332940 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:20.333231 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:20.333234 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:20.333235 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:20.333438 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:20.333549 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:20.333825 (+ 276us) server_negotiation.cc:299] Negotiation successful
0806 11:36:20.333950 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":54}
I20260806 11:36:20.334147 23913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:20.326667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:40863 (local address 127.21.14.195:59907)
0806 11:36:20.327026 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:20.327041 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:20.327211 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:20.327475 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:20.327484 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:20.327709 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:20.327850 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:20.327859 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.328778 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.328782 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:20.329357 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:20.329366 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.329851 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.329854 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:20.329950 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:20.330555 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:20.330566 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:20.330998 (+ 432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:20.332528 (+ 1530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:20.332531 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:20.332534 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:20.332748 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:20.333035 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:20.333038 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:20.333040 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:20.333131 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:20.333540 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:20.333543 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:20.333650 (+ 107us) client_negotiation.cc:769] Sending connection context
0806 11:36:20.333763 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 11:36:20.333938 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":115,"threads_started":1}
I20260806 11:36:20.337385 21563 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 11:36:20 dist-test-slave-7c35 krb5kdc[23356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016180, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 11:36:20 dist-test-slave-7c35 krb5kdc[23356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016180, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:36:20.367352 23457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:20.358101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39146 (local address 127.21.14.254:40863)
0806 11:36:20.358239 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:20.358243 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:20.358257 (+ 14us) server_negotiation.cc:407] Connection header received
0806 11:36:20.358304 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:20.358306 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:20.358349 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:20.358427 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:20.359317 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.359865 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.360988 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.361228 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.361327 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:20.363946 (+ 2619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:20.363949 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:20.363951 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:20.363985 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:20.365532 (+ 1547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:20.365967 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:20.365970 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:20.365971 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:20.366061 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:20.366378 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:20.366380 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:20.366381 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:20.366603 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:20.366711 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:20.367030 (+ 319us) server_negotiation.cc:299] Negotiation successful
0806 11:36:20.367150 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 11:36:20.370734 23402 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39146:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 11:36:20.374228 23402 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 11:36:20.392481 23926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:20.385136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.195:43505 (local address 127.0.0.1:45708)
0806 11:36:20.386101 (+ 965us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:20.386168 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:20.386304 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:20.387219 (+ 915us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:20.387226 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:20.387250 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:36:20.387410 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:20.387418 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.389021 (+ 1603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.389025 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:20.390684 (+ 1659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:20.390691 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.391796 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.391800 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:20.391951 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:20.391983 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 11:36:20.392103 (+ 120us) client_negotiation.cc:240] Negotiation successful
0806 11:36:20.392296 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":755,"mutex_wait_us":246,"thread_start_us":110,"threads_started":1}
I20260806 11:36:20.392510 23928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:20.385786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36828 (local address 127.21.14.194:33067)
0806 11:36:20.386131 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:20.386144 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:20.386763 (+ 619us) server_negotiation.cc:407] Connection header received
0806 11:36:20.386889 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:20.386897 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:20.387142 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:20.387302 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:36:20.387690 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.388905 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.390889 (+ 1984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.391560 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.391677 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:20.391846 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:20.392245 (+ 399us) server_negotiation.cc:299] Negotiation successful
0806 11:36:20.392364 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":127,"threads_started":1}
I20260806 11:36:20.392557 23930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:20.386508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42562 (local address 127.21.14.193:42373)
0806 11:36:20.386815 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:20.386821 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:20.386840 (+ 19us) server_negotiation.cc:407] Connection header received
0806 11:36:20.386941 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:20.386946 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:20.387140 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:20.387302 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:36:20.387766 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.388955 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.390481 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.391311 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.391429 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:20.391607 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:20.392310 (+ 703us) server_negotiation.cc:299] Negotiation successful
0806 11:36:20.392421 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":97,"threads_started":1}
I20260806 11:36:20.392932 23929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:20.386307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.193:42373 (local address 127.0.0.1:42562)
0806 11:36:20.386548 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:20.386561 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:20.386644 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:20.387469 (+ 825us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:20.387472 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:20.387491 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:36:20.387618 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:20.387628 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.389090 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.389093 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:20.390335 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:20.390349 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.391700 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.391707 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:20.391855 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:20.391984 (+ 129us) client_negotiation.cc:769] Sending connection context
0806 11:36:20.392105 (+ 121us) client_negotiation.cc:240] Negotiation successful
0806 11:36:20.392781 (+ 676us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"mutex_wait_us":578,"thread_start_us":121,"threads_started":1}
I20260806 11:36:20.393383 23927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:20.385205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45708 (local address 127.21.14.195:43505)
0806 11:36:20.386658 (+ 1453us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:20.386670 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:20.386697 (+ 27us) server_negotiation.cc:407] Connection header received
0806 11:36:20.386784 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:20.386789 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:20.387048 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:20.387223 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:36:20.387549 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.388876 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.390824 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.391664 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.392889 (+ 1225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:20.393098 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:20.393159 (+ 61us) server_negotiation.cc:299] Negotiation successful
0806 11:36:20.393271 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1324,"thread_start_us":117,"threads_started":1}
I20260806 11:36:20.393741 23925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:20.385186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.194:33067 (local address 127.0.0.1:36828)
0806 11:36:20.386205 (+ 1019us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:20.386217 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:20.386343 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:20.387405 (+ 1062us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:20.387408 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:20.387423 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:36:20.387571 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:20.387577 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.389075 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.389078 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:20.390678 (+ 1600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:20.390691 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.391908 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.391912 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:20.392012 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:20.392046 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 11:36:20.392124 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 11:36:20.393616 (+ 1492us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":947,"mutex_wait_us":1482,"thread_start_us":120,"threads_started":1}
I20260806 11:36:20.395890 23838 tablet_service.cc:1467] Processing CreateTablet for tablet 280a576396d9494a8efc2c431e8f5abf (DEFAULT_TABLE table=test-table [id=13626845af6d47d5923a862bfad15c79]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:36:20.395925 23548 tablet_service.cc:1467] Processing CreateTablet for tablet 280a576396d9494a8efc2c431e8f5abf (DEFAULT_TABLE table=test-table [id=13626845af6d47d5923a862bfad15c79]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:36:20.397075 23548 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 280a576396d9494a8efc2c431e8f5abf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:20.397087 23838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 280a576396d9494a8efc2c431e8f5abf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:20.399384 23694 tablet_service.cc:1467] Processing CreateTablet for tablet 280a576396d9494a8efc2c431e8f5abf (DEFAULT_TABLE table=test-table [id=13626845af6d47d5923a862bfad15c79]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:36:20.400911 23694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 280a576396d9494a8efc2c431e8f5abf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:20.404824 23931 tablet_bootstrap.cc:492] T 280a576396d9494a8efc2c431e8f5abf P 66ca4b1903f6452fa22ce688b579d7eb: Bootstrap starting.
I20260806 11:36:20.406392 23932 tablet_bootstrap.cc:492] T 280a576396d9494a8efc2c431e8f5abf P c22a8b491fc74a76a11e016052b574ab: Bootstrap starting.
I20260806 11:36:20.407335 23931 tablet_bootstrap.cc:654] T 280a576396d9494a8efc2c431e8f5abf P 66ca4b1903f6452fa22ce688b579d7eb: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:20.408243 23931 log.cc:824] T 280a576396d9494a8efc2c431e8f5abf P 66ca4b1903f6452fa22ce688b579d7eb: Log is configured to *not* fsync() on all Append() calls
I20260806 11:36:20.408941 23932 tablet_bootstrap.cc:654] T 280a576396d9494a8efc2c431e8f5abf P c22a8b491fc74a76a11e016052b574ab: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:20.409894 23932 log.cc:824] T 280a576396d9494a8efc2c431e8f5abf P c22a8b491fc74a76a11e016052b574ab: Log is configured to *not* fsync() on all Append() calls
I20260806 11:36:20.410370 23934 tablet_bootstrap.cc:492] T 280a576396d9494a8efc2c431e8f5abf P f62920d5c6274e8987acdec21c17ab85: Bootstrap starting.
I20260806 11:36:20.411208 23931 tablet_bootstrap.cc:492] T 280a576396d9494a8efc2c431e8f5abf P 66ca4b1903f6452fa22ce688b579d7eb: No bootstrap required, opened a new log
I20260806 11:36:20.411453 23931 ts_tablet_manager.cc:1397] T 280a576396d9494a8efc2c431e8f5abf P 66ca4b1903f6452fa22ce688b579d7eb: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260806 11:36:20.412532 23932 tablet_bootstrap.cc:492] T 280a576396d9494a8efc2c431e8f5abf P c22a8b491fc74a76a11e016052b574ab: No bootstrap required, opened a new log
I20260806 11:36:20.412755 23932 ts_tablet_manager.cc:1397] T 280a576396d9494a8efc2c431e8f5abf P c22a8b491fc74a76a11e016052b574ab: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260806 11:36:20.413039 23934 tablet_bootstrap.cc:654] T 280a576396d9494a8efc2c431e8f5abf P f62920d5c6274e8987acdec21c17ab85: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:20.413985 23934 log.cc:824] T 280a576396d9494a8efc2c431e8f5abf P f62920d5c6274e8987acdec21c17ab85: Log is configured to *not* fsync() on all Append() calls
I20260806 11:36:20.415875 23931 raft_consensus.cc:348] T 280a576396d9494a8efc2c431e8f5abf P 66ca4b1903f6452fa22ce688b579d7eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca4b1903f6452fa22ce688b579d7eb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 43505 } } peers { permanent_uuid: "f62920d5c6274e8987acdec21c17ab85" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 33067 } } peers { permanent_uuid: "c22a8b491fc74a76a11e016052b574ab" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42373 } }
I20260806 11:36:20.416196 23931 raft_consensus.cc:374] T 280a576396d9494a8efc2c431e8f5abf P 66ca4b1903f6452fa22ce688b579d7eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:20.416258 23931 raft_consensus.cc:729] T 280a576396d9494a8efc2c431e8f5abf P 66ca4b1903f6452fa22ce688b579d7eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66ca4b1903f6452fa22ce688b579d7eb, State: Initialized, Role: FOLLOWER
I20260806 11:36:20.416603 23934 tablet_bootstrap.cc:492] T 280a576396d9494a8efc2c431e8f5abf P f62920d5c6274e8987acdec21c17ab85: No bootstrap required, opened a new log
I20260806 11:36:20.416813 23934 ts_tablet_manager.cc:1397] T 280a576396d9494a8efc2c431e8f5abf P f62920d5c6274e8987acdec21c17ab85: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260806 11:36:20.416779 23931 consensus_queue.cc:260] T 280a576396d9494a8efc2c431e8f5abf P 66ca4b1903f6452fa22ce688b579d7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca4b1903f6452fa22ce688b579d7eb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 43505 } } peers { permanent_uuid: "f62920d5c6274e8987acdec21c17ab85" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 33067 } } peers { permanent_uuid: "c22a8b491fc74a76a11e016052b574ab" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42373 } }
I20260806 11:36:20.417529 23932 raft_consensus.cc:348] T 280a576396d9494a8efc2c431e8f5abf P c22a8b491fc74a76a11e016052b574ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca4b1903f6452fa22ce688b579d7eb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 43505 } } peers { permanent_uuid: "f62920d5c6274e8987acdec21c17ab85" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 33067 } } peers { permanent_uuid: "c22a8b491fc74a76a11e016052b574ab" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42373 } }
I20260806 11:36:20.417820 23932 raft_consensus.cc:374] T 280a576396d9494a8efc2c431e8f5abf P c22a8b491fc74a76a11e016052b574ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:20.417886 23932 raft_consensus.cc:729] T 280a576396d9494a8efc2c431e8f5abf P c22a8b491fc74a76a11e016052b574ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c22a8b491fc74a76a11e016052b574ab, State: Initialized, Role: FOLLOWER
I20260806 11:36:20.417977 23931 ts_tablet_manager.cc:1428] T 280a576396d9494a8efc2c431e8f5abf P 66ca4b1903f6452fa22ce688b579d7eb: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 11:36:20.418324 23932 consensus_queue.cc:260] T 280a576396d9494a8efc2c431e8f5abf P c22a8b491fc74a76a11e016052b574ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca4b1903f6452fa22ce688b579d7eb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 43505 } } peers { permanent_uuid: "f62920d5c6274e8987acdec21c17ab85" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 33067 } } peers { permanent_uuid: "c22a8b491fc74a76a11e016052b574ab" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42373 } }
I20260806 11:36:20.419059 23618 heartbeater.cc:503] Master 127.21.14.254:40863 was elected leader, sending a full tablet report...
I20260806 11:36:20.419344 23932 ts_tablet_manager.cc:1428] T 280a576396d9494a8efc2c431e8f5abf P c22a8b491fc74a76a11e016052b574ab: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260806 11:36:20.421018 23934 raft_consensus.cc:348] T 280a576396d9494a8efc2c431e8f5abf P f62920d5c6274e8987acdec21c17ab85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca4b1903f6452fa22ce688b579d7eb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 43505 } } peers { permanent_uuid: "f62920d5c6274e8987acdec21c17ab85" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 33067 } } peers { permanent_uuid: "c22a8b491fc74a76a11e016052b574ab" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42373 } }
I20260806 11:36:20.421316 23934 raft_consensus.cc:374] T 280a576396d9494a8efc2c431e8f5abf P f62920d5c6274e8987acdec21c17ab85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:20.421366 23934 raft_consensus.cc:729] T 280a576396d9494a8efc2c431e8f5abf P f62920d5c6274e8987acdec21c17ab85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f62920d5c6274e8987acdec21c17ab85, State: Initialized, Role: FOLLOWER
I20260806 11:36:20.421855 23934 consensus_queue.cc:260] T 280a576396d9494a8efc2c431e8f5abf P f62920d5c6274e8987acdec21c17ab85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca4b1903f6452fa22ce688b579d7eb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 43505 } } peers { permanent_uuid: "f62920d5c6274e8987acdec21c17ab85" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 33067 } } peers { permanent_uuid: "c22a8b491fc74a76a11e016052b574ab" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42373 } }
I20260806 11:36:20.422662 23762 heartbeater.cc:503] Master 127.21.14.254:40863 was elected leader, sending a full tablet report...
I20260806 11:36:20.423939 23906 heartbeater.cc:503] Master 127.21.14.254:40863 was elected leader, sending a full tablet report...
I20260806 11:36:20.426282 23934 ts_tablet_manager.cc:1428] T 280a576396d9494a8efc2c431e8f5abf P f62920d5c6274e8987acdec21c17ab85: Time spent starting tablet: real 0.009s user 0.007s sys 0.000s
I20260806 11:36:20.428048 23938 raft_consensus.cc:482] T 280a576396d9494a8efc2c431e8f5abf P c22a8b491fc74a76a11e016052b574ab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 11:36:20.428396 23938 raft_consensus.cc:504] T 280a576396d9494a8efc2c431e8f5abf P c22a8b491fc74a76a11e016052b574ab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca4b1903f6452fa22ce688b579d7eb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 43505 } } peers { permanent_uuid: "f62920d5c6274e8987acdec21c17ab85" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 33067 } } peers { permanent_uuid: "c22a8b491fc74a76a11e016052b574ab" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42373 } }
I20260806 11:36:20.429569 23938 leader_election.cc:288] T 280a576396d9494a8efc2c431e8f5abf P c22a8b491fc74a76a11e016052b574ab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66ca4b1903f6452fa22ce688b579d7eb (127.21.14.195:43505), f62920d5c6274e8987acdec21c17ab85 (127.21.14.194:33067)
I20260806 11:36:20.433660 23927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:20.429924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:41879 (local address 127.21.14.195:43505)
0806 11:36:20.430073 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:20.430090 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:20.430341 (+ 251us) server_negotiation.cc:407] Connection header received
0806 11:36:20.430505 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:20.430508 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:20.430559 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:20.430635 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:36:20.430926 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.431489 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.432616 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.433070 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.433204 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:20.433261 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:20.433442 (+ 181us) server_negotiation.cc:299] Negotiation successful
0806 11:36:20.433519 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 11:36:20.433691 23941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:20.429742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.195:43505 (local address 127.21.14.193:41879)
0806 11:36:20.430235 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:20.430249 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:20.430392 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:20.430659 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:20.430662 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:20.430679 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:36:20.430784 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:20.430788 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.431633 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.431636 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:20.432475 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:20.432484 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.433199 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.433203 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:20.433307 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:20.433334 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 11:36:20.433424 (+ 90us) client_negotiation.cc:240] Negotiation successful
0806 11:36:20.433506 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":398,"mutex_wait_us":1,"spinlock_wait_cycles":10112,"thread_start_us":168,"threads_started":1}
I20260806 11:36:20.434568 23844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "280a576396d9494a8efc2c431e8f5abf" candidate_uuid: "c22a8b491fc74a76a11e016052b574ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66ca4b1903f6452fa22ce688b579d7eb" is_pre_election: true
I20260806 11:36:20.434819 23940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:20.429862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.194:33067 (local address 127.21.14.193:52851)
0806 11:36:20.430252 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:20.430264 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:20.430392 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:20.431032 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:20.431035 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:20.431052 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:36:20.431167 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:20.431172 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.432215 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.432218 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:20.433436 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:20.433446 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.434430 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.434433 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:20.434521 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:20.434537 (+ 16us) client_negotiation.cc:769] Sending connection context
0806 11:36:20.434595 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 11:36:20.434651 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"spinlock_wait_cycles":22656,"thread_start_us":137,"threads_started":1}
I20260806 11:36:20.434885 23928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:20.429925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:52851 (local address 127.21.14.194:33067)
0806 11:36:20.430089 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:20.430093 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:20.430379 (+ 286us) server_negotiation.cc:407] Connection header received
0806 11:36:20.430766 (+ 387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:20.430769 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:20.430812 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:20.430925 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:36:20.431286 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.432070 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.433584 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.434171 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.434276 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:20.434325 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:20.434653 (+ 328us) server_negotiation.cc:299] Negotiation successful
0806 11:36:20.434747 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 11:36:20.435024 23844 raft_consensus.cc:2454] T 280a576396d9494a8efc2c431e8f5abf P 66ca4b1903f6452fa22ce688b579d7eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c22a8b491fc74a76a11e016052b574ab in term 0.
I20260806 11:36:20.435606 23501 leader_election.cc:302] T 280a576396d9494a8efc2c431e8f5abf P c22a8b491fc74a76a11e016052b574ab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66ca4b1903f6452fa22ce688b579d7eb, c22a8b491fc74a76a11e016052b574ab; no voters:
I20260806 11:36:20.435737 23705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "280a576396d9494a8efc2c431e8f5abf" candidate_uuid: "c22a8b491fc74a76a11e016052b574ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f62920d5c6274e8987acdec21c17ab85" is_pre_election: true
I20260806 11:36:20.436161 23938 raft_consensus.cc:2781] T 280a576396d9494a8efc2c431e8f5abf P c22a8b491fc74a76a11e016052b574ab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 11:36:20.436276 23938 raft_consensus.cc:482] T 280a576396d9494a8efc2c431e8f5abf P c22a8b491fc74a76a11e016052b574ab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 11:36:20.436335 23938 raft_consensus.cc:3037] T 280a576396d9494a8efc2c431e8f5abf P c22a8b491fc74a76a11e016052b574ab [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:36:20.436285 23705 raft_consensus.cc:2454] T 280a576396d9494a8efc2c431e8f5abf P f62920d5c6274e8987acdec21c17ab85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c22a8b491fc74a76a11e016052b574ab in term 0.
I20260806 11:36:20.437564 23938 raft_consensus.cc:504] T 280a576396d9494a8efc2c431e8f5abf P c22a8b491fc74a76a11e016052b574ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca4b1903f6452fa22ce688b579d7eb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 43505 } } peers { permanent_uuid: "f62920d5c6274e8987acdec21c17ab85" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 33067 } } peers { permanent_uuid: "c22a8b491fc74a76a11e016052b574ab" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42373 } }
I20260806 11:36:20.437947 23938 leader_election.cc:288] T 280a576396d9494a8efc2c431e8f5abf P c22a8b491fc74a76a11e016052b574ab [CANDIDATE]: Term 1 election: Requested vote from peers 66ca4b1903f6452fa22ce688b579d7eb (127.21.14.195:43505), f62920d5c6274e8987acdec21c17ab85 (127.21.14.194:33067)
I20260806 11:36:20.438480 23844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "280a576396d9494a8efc2c431e8f5abf" candidate_uuid: "c22a8b491fc74a76a11e016052b574ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66ca4b1903f6452fa22ce688b579d7eb"
I20260806 11:36:20.438558 23705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "280a576396d9494a8efc2c431e8f5abf" candidate_uuid: "c22a8b491fc74a76a11e016052b574ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f62920d5c6274e8987acdec21c17ab85"
I20260806 11:36:20.438640 23844 raft_consensus.cc:3037] T 280a576396d9494a8efc2c431e8f5abf P 66ca4b1903f6452fa22ce688b579d7eb [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:36:20.438714 23705 raft_consensus.cc:3037] T 280a576396d9494a8efc2c431e8f5abf P f62920d5c6274e8987acdec21c17ab85 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:36:20.439874 23844 raft_consensus.cc:2454] T 280a576396d9494a8efc2c431e8f5abf P 66ca4b1903f6452fa22ce688b579d7eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c22a8b491fc74a76a11e016052b574ab in term 1.
I20260806 11:36:20.439877 23705 raft_consensus.cc:2454] T 280a576396d9494a8efc2c431e8f5abf P f62920d5c6274e8987acdec21c17ab85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c22a8b491fc74a76a11e016052b574ab in term 1.
I20260806 11:36:20.440299 23500 leader_election.cc:302] T 280a576396d9494a8efc2c431e8f5abf P c22a8b491fc74a76a11e016052b574ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 66ca4b1903f6452fa22ce688b579d7eb, c22a8b491fc74a76a11e016052b574ab, f62920d5c6274e8987acdec21c17ab85; no voters:
I20260806 11:36:20.440513 23938 raft_consensus.cc:2781] T 280a576396d9494a8efc2c431e8f5abf P c22a8b491fc74a76a11e016052b574ab [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:36:20.440768 23938 raft_consensus.cc:686] T 280a576396d9494a8efc2c431e8f5abf P c22a8b491fc74a76a11e016052b574ab [term 1 LEADER]: Becoming Leader. State: Replica: c22a8b491fc74a76a11e016052b574ab, State: Running, Role: LEADER
I20260806 11:36:20.441181 23938 consensus_queue.cc:237] T 280a576396d9494a8efc2c431e8f5abf P c22a8b491fc74a76a11e016052b574ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca4b1903f6452fa22ce688b579d7eb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 43505 } } peers { permanent_uuid: "f62920d5c6274e8987acdec21c17ab85" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 33067 } } peers { permanent_uuid: "c22a8b491fc74a76a11e016052b574ab" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42373 } }
I20260806 11:36:20.444840 23401 catalog_manager.cc:5495] T 280a576396d9494a8efc2c431e8f5abf P c22a8b491fc74a76a11e016052b574ab reported cstate change: term changed from 0 to 1, leader changed from <none> to c22a8b491fc74a76a11e016052b574ab (127.21.14.193). New cstate: current_term: 1 leader_uuid: "c22a8b491fc74a76a11e016052b574ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca4b1903f6452fa22ce688b579d7eb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 43505 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f62920d5c6274e8987acdec21c17ab85" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 33067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c22a8b491fc74a76a11e016052b574ab" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42373 } health_report { overall_health: HEALTHY } } }
Aug 06 11:36:20 dist-test-slave-7c35 krb5kdc[23356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016179, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.254@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:36:20.503387 23951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:20.492975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:40863 (local address 127.0.0.1:39156)
0806 11:36:20.493356 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:20.493376 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:20.493516 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:20.493795 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:20.493797 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:20.494246 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:20.494412 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:20.494422 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.495379 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.495385 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:20.496369 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:20.496386 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.497095 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.497100 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:20.497239 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:20.497852 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:20.497866 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:20.499664 (+ 1798us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:20.501605 (+ 1941us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:20.501619 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:20.501626 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:20.501916 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:20.502224 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:20.502226 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:20.502228 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:20.502318 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:20.502767 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:20.502775 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:20.502967 (+ 192us) client_negotiation.cc:769] Sending connection context
0806 11:36:20.503080 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 11:36:20.503234 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"mutex_wait_us":23,"thread_start_us":165,"threads_started":1}
I20260806 11:36:20.503449 23457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:20.493005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39156 (local address 127.21.14.254:40863)
0806 11:36:20.493161 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:20.493165 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:20.493483 (+ 318us) server_negotiation.cc:407] Connection header received
0806 11:36:20.493631 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:20.493634 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:20.493680 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:20.493751 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:20.494559 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.495222 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.496544 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.496959 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.497086 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:20.499863 (+ 2777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:20.499867 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:20.499869 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:20.499897 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:20.501457 (+ 1560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:20.502031 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:20.502034 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:20.502036 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:20.502130 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:20.502434 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:20.502437 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:20.502439 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:20.502664 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:20.502754 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:20.503146 (+ 392us) server_negotiation.cc:299] Negotiation successful
0806 11:36:20.503309 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":34,"server-negotiator.queue_time_us":59}
I20260806 11:36:20.508025 23401 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.21.14.254@KRBTEST.COM'} at 127.0.0.1:39156:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 11:36:20.508965 23401 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 11:36:20.514714 23548 tablet_service.cc:1467] Processing CreateTablet for tablet 00e347bf3f0c4f48b54f0b73d248979d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=959ef30ac6c34db0875fc3c8ac6d5a81]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 11:36:20.514711 23694 tablet_service.cc:1467] Processing CreateTablet for tablet 00e347bf3f0c4f48b54f0b73d248979d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=959ef30ac6c34db0875fc3c8ac6d5a81]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 11:36:20.515084 23838 tablet_service.cc:1467] Processing CreateTablet for tablet 00e347bf3f0c4f48b54f0b73d248979d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=959ef30ac6c34db0875fc3c8ac6d5a81]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 11:36:20.515580 23548 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00e347bf3f0c4f48b54f0b73d248979d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:20.515585 23694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00e347bf3f0c4f48b54f0b73d248979d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:20.515697 23838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00e347bf3f0c4f48b54f0b73d248979d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:20.518357 23932 tablet_bootstrap.cc:492] T 00e347bf3f0c4f48b54f0b73d248979d P c22a8b491fc74a76a11e016052b574ab: Bootstrap starting.
I20260806 11:36:20.518884 23931 tablet_bootstrap.cc:492] T 00e347bf3f0c4f48b54f0b73d248979d P 66ca4b1903f6452fa22ce688b579d7eb: Bootstrap starting.
I20260806 11:36:20.519593 23932 tablet_bootstrap.cc:654] T 00e347bf3f0c4f48b54f0b73d248979d P c22a8b491fc74a76a11e016052b574ab: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:20.519670 23934 tablet_bootstrap.cc:492] T 00e347bf3f0c4f48b54f0b73d248979d P f62920d5c6274e8987acdec21c17ab85: Bootstrap starting.
I20260806 11:36:20.519927 23931 tablet_bootstrap.cc:654] T 00e347bf3f0c4f48b54f0b73d248979d P 66ca4b1903f6452fa22ce688b579d7eb: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:20.520913 23934 tablet_bootstrap.cc:654] T 00e347bf3f0c4f48b54f0b73d248979d P f62920d5c6274e8987acdec21c17ab85: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:20.521363 23931 tablet_bootstrap.cc:492] T 00e347bf3f0c4f48b54f0b73d248979d P 66ca4b1903f6452fa22ce688b579d7eb: No bootstrap required, opened a new log
I20260806 11:36:20.521479 23931 ts_tablet_manager.cc:1397] T 00e347bf3f0c4f48b54f0b73d248979d P 66ca4b1903f6452fa22ce688b579d7eb: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 11:36:20.521821 23932 tablet_bootstrap.cc:492] T 00e347bf3f0c4f48b54f0b73d248979d P c22a8b491fc74a76a11e016052b574ab: No bootstrap required, opened a new log
I20260806 11:36:20.521922 23932 ts_tablet_manager.cc:1397] T 00e347bf3f0c4f48b54f0b73d248979d P c22a8b491fc74a76a11e016052b574ab: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260806 11:36:20.522022 23931 raft_consensus.cc:348] T 00e347bf3f0c4f48b54f0b73d248979d P 66ca4b1903f6452fa22ce688b579d7eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c22a8b491fc74a76a11e016052b574ab" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42373 } } peers { permanent_uuid: "f62920d5c6274e8987acdec21c17ab85" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 33067 } } peers { permanent_uuid: "66ca4b1903f6452fa22ce688b579d7eb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 43505 } }
I20260806 11:36:20.522151 23931 raft_consensus.cc:374] T 00e347bf3f0c4f48b54f0b73d248979d P 66ca4b1903f6452fa22ce688b579d7eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:20.522182 23931 raft_consensus.cc:729] T 00e347bf3f0c4f48b54f0b73d248979d P 66ca4b1903f6452fa22ce688b579d7eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66ca4b1903f6452fa22ce688b579d7eb, State: Initialized, Role: FOLLOWER
I20260806 11:36:20.522240 23934 tablet_bootstrap.cc:492] T 00e347bf3f0c4f48b54f0b73d248979d P f62920d5c6274e8987acdec21c17ab85: No bootstrap required, opened a new log
I20260806 11:36:20.522327 23934 ts_tablet_manager.cc:1397] T 00e347bf3f0c4f48b54f0b73d248979d P f62920d5c6274e8987acdec21c17ab85: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 11:36:20.522493 23932 raft_consensus.cc:348] T 00e347bf3f0c4f48b54f0b73d248979d P c22a8b491fc74a76a11e016052b574ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c22a8b491fc74a76a11e016052b574ab" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42373 } } peers { permanent_uuid: "f62920d5c6274e8987acdec21c17ab85" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 33067 } } peers { permanent_uuid: "66ca4b1903f6452fa22ce688b579d7eb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 43505 } }
I20260806 11:36:20.522626 23932 raft_consensus.cc:374] T 00e347bf3f0c4f48b54f0b73d248979d P c22a8b491fc74a76a11e016052b574ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:20.522667 23932 raft_consensus.cc:729] T 00e347bf3f0c4f48b54f0b73d248979d P c22a8b491fc74a76a11e016052b574ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c22a8b491fc74a76a11e016052b574ab, State: Initialized, Role: FOLLOWER
I20260806 11:36:20.522604 23931 consensus_queue.cc:260] T 00e347bf3f0c4f48b54f0b73d248979d P 66ca4b1903f6452fa22ce688b579d7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c22a8b491fc74a76a11e016052b574ab" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42373 } } peers { permanent_uuid: "f62920d5c6274e8987acdec21c17ab85" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 33067 } } peers { permanent_uuid: "66ca4b1903f6452fa22ce688b579d7eb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 43505 } }
I20260806 11:36:20.522804 23932 consensus_queue.cc:260] T 00e347bf3f0c4f48b54f0b73d248979d P c22a8b491fc74a76a11e016052b574ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c22a8b491fc74a76a11e016052b574ab" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42373 } } peers { permanent_uuid: "f62920d5c6274e8987acdec21c17ab85" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 33067 } } peers { permanent_uuid: "66ca4b1903f6452fa22ce688b579d7eb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 43505 } }
I20260806 11:36:20.523012 23937 tablet_replica.cc:439] T 00e347bf3f0c4f48b54f0b73d248979d P 66ca4b1903f6452fa22ce688b579d7eb: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c22a8b491fc74a76a11e016052b574ab" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42373 } } peers { permanent_uuid: "f62920d5c6274e8987acdec21c17ab85" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 33067 } } peers { permanent_uuid: "66ca4b1903f6452fa22ce688b579d7eb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 43505 } } }
I20260806 11:36:20.523095 23938 tablet_replica.cc:439] T 00e347bf3f0c4f48b54f0b73d248979d P c22a8b491fc74a76a11e016052b574ab: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c22a8b491fc74a76a11e016052b574ab" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42373 } } peers { permanent_uuid: "f62920d5c6274e8987acdec21c17ab85" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 33067 } } peers { permanent_uuid: "66ca4b1903f6452fa22ce688b579d7eb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 43505 } } }
I20260806 11:36:20.523183 23937 tablet_replica.cc:442] T 00e347bf3f0c4f48b54f0b73d248979d P 66ca4b1903f6452fa22ce688b579d7eb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 11:36:20.523222 23938 tablet_replica.cc:442] T 00e347bf3f0c4f48b54f0b73d248979d P c22a8b491fc74a76a11e016052b574ab: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 11:36:20.523439 23931 ts_tablet_manager.cc:1428] T 00e347bf3f0c4f48b54f0b73d248979d P 66ca4b1903f6452fa22ce688b579d7eb: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260806 11:36:20.523108 23932 ts_tablet_manager.cc:1428] T 00e347bf3f0c4f48b54f0b73d248979d P c22a8b491fc74a76a11e016052b574ab: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260806 11:36:20.522835 23934 raft_consensus.cc:348] T 00e347bf3f0c4f48b54f0b73d248979d P f62920d5c6274e8987acdec21c17ab85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c22a8b491fc74a76a11e016052b574ab" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42373 } } peers { permanent_uuid: "f62920d5c6274e8987acdec21c17ab85" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 33067 } } peers { permanent_uuid: "66ca4b1903f6452fa22ce688b579d7eb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 43505 } }
I20260806 11:36:20.523919 23934 raft_consensus.cc:374] T 00e347bf3f0c4f48b54f0b73d248979d P f62920d5c6274e8987acdec21c17ab85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:20.523983 23934 raft_consensus.cc:729] T 00e347bf3f0c4f48b54f0b73d248979d P f62920d5c6274e8987acdec21c17ab85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f62920d5c6274e8987acdec21c17ab85, State: Initialized, Role: FOLLOWER
I20260806 11:36:20.524139 23934 consensus_queue.cc:260] T 00e347bf3f0c4f48b54f0b73d248979d P f62920d5c6274e8987acdec21c17ab85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c22a8b491fc74a76a11e016052b574ab" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42373 } } peers { permanent_uuid: "f62920d5c6274e8987acdec21c17ab85" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 33067 } } peers { permanent_uuid: "66ca4b1903f6452fa22ce688b579d7eb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 43505 } }
I20260806 11:36:20.524441 23934 ts_tablet_manager.cc:1428] T 00e347bf3f0c4f48b54f0b73d248979d P f62920d5c6274e8987acdec21c17ab85: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260806 11:36:20.524480 23939 tablet_replica.cc:439] T 00e347bf3f0c4f48b54f0b73d248979d P f62920d5c6274e8987acdec21c17ab85: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c22a8b491fc74a76a11e016052b574ab" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42373 } } peers { permanent_uuid: "f62920d5c6274e8987acdec21c17ab85" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 33067 } } peers { permanent_uuid: "66ca4b1903f6452fa22ce688b579d7eb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 43505 } } }
I20260806 11:36:20.524646 23939 tablet_replica.cc:442] T 00e347bf3f0c4f48b54f0b73d248979d P f62920d5c6274e8987acdec21c17ab85: This TxnStatusTablet replica's current role is: FOLLOWER
W20260806 11:36:20.543543 23619 tablet.cc:2369] T 00e347bf3f0c4f48b54f0b73d248979d P c22a8b491fc74a76a11e016052b574ab: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 11:36:20.548523 23939 raft_consensus.cc:482] T 00e347bf3f0c4f48b54f0b73d248979d P f62920d5c6274e8987acdec21c17ab85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 11:36:20.548686 23939 raft_consensus.cc:504] T 00e347bf3f0c4f48b54f0b73d248979d P f62920d5c6274e8987acdec21c17ab85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c22a8b491fc74a76a11e016052b574ab" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42373 } } peers { permanent_uuid: "f62920d5c6274e8987acdec21c17ab85" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 33067 } } peers { permanent_uuid: "66ca4b1903f6452fa22ce688b579d7eb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 43505 } }
I20260806 11:36:20.549962 23939 leader_election.cc:288] T 00e347bf3f0c4f48b54f0b73d248979d P f62920d5c6274e8987acdec21c17ab85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c22a8b491fc74a76a11e016052b574ab (127.21.14.193:42373), 66ca4b1903f6452fa22ce688b579d7eb (127.21.14.195:43505)
W20260806 11:36:20.553570 23763 tablet.cc:2369] T 00e347bf3f0c4f48b54f0b73d248979d P f62920d5c6274e8987acdec21c17ab85: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 11:36:20.553959 23653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:20.550276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.195:43505 (local address 127.21.14.194:38839)
0806 11:36:20.550436 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:20.550451 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:20.550554 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:20.550840 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:20.550861 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:20.550878 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:36:20.551006 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:20.551012 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.551906 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.551912 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:20.552717 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:20.552726 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.553495 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.553498 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:20.553615 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:20.553639 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 11:36:20.553715 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 11:36:20.553792 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260806 11:36:20.554380 23927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:20.550396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:38839 (local address 127.21.14.195:43505)
0806 11:36:20.550531 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:20.550536 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:20.550554 (+ 18us) server_negotiation.cc:407] Connection header received
0806 11:36:20.550662 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:20.550666 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:20.550718 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:20.550811 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:36:20.551138 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.551797 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.552856 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.553373 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.553506 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:20.553563 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:20.554210 (+ 647us) server_negotiation.cc:299] Negotiation successful
0806 11:36:20.554286 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 11:36:20.554935 23930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:20.550456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:43317 (local address 127.21.14.193:42373)
0806 11:36:20.550627 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:20.550631 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:20.550969 (+ 338us) server_negotiation.cc:407] Connection header received
0806 11:36:20.551183 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:20.551186 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:20.551235 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:20.551318 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:36:20.551656 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.552152 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.553363 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.553937 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.554074 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:20.554123 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:20.554709 (+ 586us) server_negotiation.cc:299] Negotiation successful
0806 11:36:20.554796 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 11:36:20.554972 23955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:20.550290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.193:42373 (local address 127.21.14.194:43317)
0806 11:36:20.550839 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:20.550891 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:20.551005 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:20.551331 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:20.551334 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:20.551349 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:36:20.551515 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:20.551521 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.552262 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.552268 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:20.553218 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:20.553228 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.554450 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.554454 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:20.554568 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:20.554599 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 11:36:20.554675 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 11:36:20.554750 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"mutex_wait_us":151,"thread_start_us":177,"threads_started":1}
I20260806 11:36:20.554920 23844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00e347bf3f0c4f48b54f0b73d248979d" candidate_uuid: "f62920d5c6274e8987acdec21c17ab85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66ca4b1903f6452fa22ce688b579d7eb" is_pre_election: true
I20260806 11:36:20.555114 23844 raft_consensus.cc:2454] T 00e347bf3f0c4f48b54f0b73d248979d P 66ca4b1903f6452fa22ce688b579d7eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f62920d5c6274e8987acdec21c17ab85 in term 0.
I20260806 11:36:20.555569 23648 leader_election.cc:302] T 00e347bf3f0c4f48b54f0b73d248979d P f62920d5c6274e8987acdec21c17ab85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66ca4b1903f6452fa22ce688b579d7eb, f62920d5c6274e8987acdec21c17ab85; no voters:
I20260806 11:36:20.555569 23569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00e347bf3f0c4f48b54f0b73d248979d" candidate_uuid: "f62920d5c6274e8987acdec21c17ab85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c22a8b491fc74a76a11e016052b574ab" is_pre_election: true
I20260806 11:36:20.555822 23569 raft_consensus.cc:2454] T 00e347bf3f0c4f48b54f0b73d248979d P c22a8b491fc74a76a11e016052b574ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f62920d5c6274e8987acdec21c17ab85 in term 0.
I20260806 11:36:20.555985 23939 raft_consensus.cc:2781] T 00e347bf3f0c4f48b54f0b73d248979d P f62920d5c6274e8987acdec21c17ab85 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 11:36:20.556089 23939 raft_consensus.cc:482] T 00e347bf3f0c4f48b54f0b73d248979d P f62920d5c6274e8987acdec21c17ab85 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 11:36:20.556149 23939 raft_consensus.cc:3037] T 00e347bf3f0c4f48b54f0b73d248979d P f62920d5c6274e8987acdec21c17ab85 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:36:20.557238 23939 raft_consensus.cc:504] T 00e347bf3f0c4f48b54f0b73d248979d P f62920d5c6274e8987acdec21c17ab85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c22a8b491fc74a76a11e016052b574ab" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42373 } } peers { permanent_uuid: "f62920d5c6274e8987acdec21c17ab85" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 33067 } } peers { permanent_uuid: "66ca4b1903f6452fa22ce688b579d7eb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 43505 } }
I20260806 11:36:20.557682 23939 leader_election.cc:288] T 00e347bf3f0c4f48b54f0b73d248979d P f62920d5c6274e8987acdec21c17ab85 [CANDIDATE]: Term 1 election: Requested vote from peers c22a8b491fc74a76a11e016052b574ab (127.21.14.193:42373), 66ca4b1903f6452fa22ce688b579d7eb (127.21.14.195:43505)
I20260806 11:36:20.558076 23844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00e347bf3f0c4f48b54f0b73d248979d" candidate_uuid: "f62920d5c6274e8987acdec21c17ab85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66ca4b1903f6452fa22ce688b579d7eb"
I20260806 11:36:20.558064 23569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00e347bf3f0c4f48b54f0b73d248979d" candidate_uuid: "f62920d5c6274e8987acdec21c17ab85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c22a8b491fc74a76a11e016052b574ab"
I20260806 11:36:20.558200 23569 raft_consensus.cc:3037] T 00e347bf3f0c4f48b54f0b73d248979d P c22a8b491fc74a76a11e016052b574ab [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:36:20.558200 23844 raft_consensus.cc:3037] T 00e347bf3f0c4f48b54f0b73d248979d P 66ca4b1903f6452fa22ce688b579d7eb [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:36:20.559794 23844 raft_consensus.cc:2454] T 00e347bf3f0c4f48b54f0b73d248979d P 66ca4b1903f6452fa22ce688b579d7eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f62920d5c6274e8987acdec21c17ab85 in term 1.
I20260806 11:36:20.559794 23569 raft_consensus.cc:2454] T 00e347bf3f0c4f48b54f0b73d248979d P c22a8b491fc74a76a11e016052b574ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f62920d5c6274e8987acdec21c17ab85 in term 1.
I20260806 11:36:20.560189 23648 leader_election.cc:302] T 00e347bf3f0c4f48b54f0b73d248979d P f62920d5c6274e8987acdec21c17ab85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 66ca4b1903f6452fa22ce688b579d7eb, c22a8b491fc74a76a11e016052b574ab, f62920d5c6274e8987acdec21c17ab85; no voters:
I20260806 11:36:20.560516 23939 raft_consensus.cc:2781] T 00e347bf3f0c4f48b54f0b73d248979d P f62920d5c6274e8987acdec21c17ab85 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:36:20.560729 23939 raft_consensus.cc:686] T 00e347bf3f0c4f48b54f0b73d248979d P f62920d5c6274e8987acdec21c17ab85 [term 1 LEADER]: Becoming Leader. State: Replica: f62920d5c6274e8987acdec21c17ab85, State: Running, Role: LEADER
I20260806 11:36:20.561131 23939 consensus_queue.cc:237] T 00e347bf3f0c4f48b54f0b73d248979d P f62920d5c6274e8987acdec21c17ab85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c22a8b491fc74a76a11e016052b574ab" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42373 } } peers { permanent_uuid: "f62920d5c6274e8987acdec21c17ab85" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 33067 } } peers { permanent_uuid: "66ca4b1903f6452fa22ce688b579d7eb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 43505 } }
I20260806 11:36:20.563874 23956 tablet_replica.cc:439] T 00e347bf3f0c4f48b54f0b73d248979d P f62920d5c6274e8987acdec21c17ab85: TxnStatusTablet state changed. Reason: New leader f62920d5c6274e8987acdec21c17ab85. Latest consensus state: current_term: 1 leader_uuid: "f62920d5c6274e8987acdec21c17ab85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c22a8b491fc74a76a11e016052b574ab" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42373 } } peers { permanent_uuid: "f62920d5c6274e8987acdec21c17ab85" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 33067 } } peers { permanent_uuid: "66ca4b1903f6452fa22ce688b579d7eb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 43505 } } }
I20260806 11:36:20.564020 23956 tablet_replica.cc:442] T 00e347bf3f0c4f48b54f0b73d248979d P f62920d5c6274e8987acdec21c17ab85: This TxnStatusTablet replica's current role is: LEADER
I20260806 11:36:20.564435 23958 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 11:36:20.564689 23958 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 11:36:20.564985 23402 catalog_manager.cc:5495] T 00e347bf3f0c4f48b54f0b73d248979d P f62920d5c6274e8987acdec21c17ab85 reported cstate change: term changed from 0 to 1, leader changed from <none> to f62920d5c6274e8987acdec21c17ab85 (127.21.14.194). New cstate: current_term: 1 leader_uuid: "f62920d5c6274e8987acdec21c17ab85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c22a8b491fc74a76a11e016052b574ab" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f62920d5c6274e8987acdec21c17ab85" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 33067 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "66ca4b1903f6452fa22ce688b579d7eb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 43505 } health_report { overall_health: UNKNOWN } } }
W20260806 11:36:20.570150 23908 tablet.cc:2369] T 00e347bf3f0c4f48b54f0b73d248979d P 66ca4b1903f6452fa22ce688b579d7eb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 11:36:20.584671 23951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:20.580283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.194:33067 (local address 127.0.0.1:36838)
0806 11:36:20.580435 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:20.580453 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:20.580571 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:20.580927 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:20.580929 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:20.580936 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 11:36:20.581035 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:20.581040 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.582065 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.582067 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:20.582690 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:20.582698 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.583304 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.583307 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:20.583382 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:20.583436 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 11:36:20.584308 (+ 872us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 11:36:20.584340 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 11:36:20.584412 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 11:36:20.584547 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260806 11:36:20.584703 23928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:20.580398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36838 (local address 127.21.14.194:33067)
0806 11:36:20.580534 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:20.580539 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:20.580557 (+ 18us) server_negotiation.cc:407] Connection header received
0806 11:36:20.580728 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:20.580731 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:20.580802 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:20.580924 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 11:36:20.581134 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.581935 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.582802 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.583194 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.583299 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:20.583599 (+ 300us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 11:36:20.584187 (+ 588us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 11:36:20.584300 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:20.584466 (+ 166us) server_negotiation.cc:299] Negotiation successful
0806 11:36:20.584530 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 11:36:20.590524 23844 raft_consensus.cc:1261] T 00e347bf3f0c4f48b54f0b73d248979d P 66ca4b1903f6452fa22ce688b579d7eb [term 1 FOLLOWER]: Refusing update from remote peer f62920d5c6274e8987acdec21c17ab85: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 11:36:20.591424 23569 raft_consensus.cc:1261] T 00e347bf3f0c4f48b54f0b73d248979d P c22a8b491fc74a76a11e016052b574ab [term 1 FOLLOWER]: Refusing update from remote peer f62920d5c6274e8987acdec21c17ab85: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 11:36:20.591420 23956 consensus_queue.cc:1035] T 00e347bf3f0c4f48b54f0b73d248979d P f62920d5c6274e8987acdec21c17ab85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66ca4b1903f6452fa22ce688b579d7eb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 43505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 11:36:20.592023 23956 consensus_queue.cc:1035] T 00e347bf3f0c4f48b54f0b73d248979d P f62920d5c6274e8987acdec21c17ab85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c22a8b491fc74a76a11e016052b574ab" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 11:36:20.596437 23938 tablet_replica.cc:439] T 00e347bf3f0c4f48b54f0b73d248979d P c22a8b491fc74a76a11e016052b574ab: TxnStatusTablet state changed. Reason: New leader f62920d5c6274e8987acdec21c17ab85. Latest consensus state: current_term: 1 leader_uuid: "f62920d5c6274e8987acdec21c17ab85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c22a8b491fc74a76a11e016052b574ab" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42373 } } peers { permanent_uuid: "f62920d5c6274e8987acdec21c17ab85" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 33067 } } peers { permanent_uuid: "66ca4b1903f6452fa22ce688b579d7eb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 43505 } } }
I20260806 11:36:20.596429 23937 tablet_replica.cc:439] T 00e347bf3f0c4f48b54f0b73d248979d P 66ca4b1903f6452fa22ce688b579d7eb: TxnStatusTablet state changed. Reason: New leader f62920d5c6274e8987acdec21c17ab85. Latest consensus state: current_term: 1 leader_uuid: "f62920d5c6274e8987acdec21c17ab85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c22a8b491fc74a76a11e016052b574ab" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42373 } } peers { permanent_uuid: "f62920d5c6274e8987acdec21c17ab85" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 33067 } } peers { permanent_uuid: "66ca4b1903f6452fa22ce688b579d7eb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 43505 } } }
I20260806 11:36:20.596611 23938 tablet_replica.cc:442] T 00e347bf3f0c4f48b54f0b73d248979d P c22a8b491fc74a76a11e016052b574ab: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 11:36:20.596616 23937 tablet_replica.cc:442] T 00e347bf3f0c4f48b54f0b73d248979d P 66ca4b1903f6452fa22ce688b579d7eb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 11:36:20.601328 23956 tablet_replica.cc:439] T 00e347bf3f0c4f48b54f0b73d248979d P f62920d5c6274e8987acdec21c17ab85: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f62920d5c6274e8987acdec21c17ab85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c22a8b491fc74a76a11e016052b574ab" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42373 } } peers { permanent_uuid: "f62920d5c6274e8987acdec21c17ab85" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 33067 } } peers { permanent_uuid: "66ca4b1903f6452fa22ce688b579d7eb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 43505 } } }
I20260806 11:36:20.601488 23956 tablet_replica.cc:442] T 00e347bf3f0c4f48b54f0b73d248979d P f62920d5c6274e8987acdec21c17ab85: This TxnStatusTablet replica's current role is: LEADER
I20260806 11:36:20.601624 23956 tablet_replica.cc:439] T 00e347bf3f0c4f48b54f0b73d248979d P f62920d5c6274e8987acdec21c17ab85: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f62920d5c6274e8987acdec21c17ab85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c22a8b491fc74a76a11e016052b574ab" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42373 } } peers { permanent_uuid: "f62920d5c6274e8987acdec21c17ab85" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 33067 } } peers { permanent_uuid: "66ca4b1903f6452fa22ce688b579d7eb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 43505 } } }
I20260806 11:36:20.601698 23956 tablet_replica.cc:442] T 00e347bf3f0c4f48b54f0b73d248979d P f62920d5c6274e8987acdec21c17ab85: This TxnStatusTablet replica's current role is: LEADER
I20260806 11:36:20.601814 23961 mvcc.cc:205] Tried to move back new op lower bound from 7315522275691073536 to 7315522275581644800. Current Snapshot: MvccSnapshot[applied={T|T < 7315522275691073536}]
I20260806 11:36:20.602605 23938 tablet_replica.cc:439] T 00e347bf3f0c4f48b54f0b73d248979d P c22a8b491fc74a76a11e016052b574ab: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f62920d5c6274e8987acdec21c17ab85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c22a8b491fc74a76a11e016052b574ab" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42373 } } peers { permanent_uuid: "f62920d5c6274e8987acdec21c17ab85" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 33067 } } peers { permanent_uuid: "66ca4b1903f6452fa22ce688b579d7eb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 43505 } } }
I20260806 11:36:20.602736 23938 tablet_replica.cc:442] T 00e347bf3f0c4f48b54f0b73d248979d P c22a8b491fc74a76a11e016052b574ab: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 11:36:20.603499 23937 tablet_replica.cc:439] T 00e347bf3f0c4f48b54f0b73d248979d P 66ca4b1903f6452fa22ce688b579d7eb: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f62920d5c6274e8987acdec21c17ab85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c22a8b491fc74a76a11e016052b574ab" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42373 } } peers { permanent_uuid: "f62920d5c6274e8987acdec21c17ab85" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 33067 } } peers { permanent_uuid: "66ca4b1903f6452fa22ce688b579d7eb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 43505 } } }
I20260806 11:36:20.603618 23937 tablet_replica.cc:442] T 00e347bf3f0c4f48b54f0b73d248979d P 66ca4b1903f6452fa22ce688b579d7eb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 11:36:20.606477 23963 mvcc.cc:205] Tried to move back new op lower bound from 7315522275691073536 to 7315522275581644800. Current Snapshot: MvccSnapshot[applied={T|T < 7315522275691073536}]
I20260806 11:36:20.606698 23962 mvcc.cc:205] Tried to move back new op lower bound from 7315522275691073536 to 7315522275581644800. Current Snapshot: MvccSnapshot[applied={T|T < 7315522275691073536}]
I20260806 11:36:20.613817 23930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:20.609683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42566 (local address 127.21.14.193:42373)
0806 11:36:20.609804 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:20.609808 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:20.609845 (+ 37us) server_negotiation.cc:407] Connection header received
0806 11:36:20.610027 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:20.610032 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:20.610114 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:20.610229 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 11:36:20.610490 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.611174 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.612186 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.612458 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.612574 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:20.612872 (+ 298us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 11:36:20.613362 (+ 490us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 11:36:20.613499 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:20.613562 (+ 63us) server_negotiation.cc:299] Negotiation successful
0806 11:36:20.613627 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 11:36:20.628628 23973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:20.624070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.194:33067 (local address 127.21.14.193:55227)
0806 11:36:20.624500 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:20.624515 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:20.624625 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:20.624929 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:20.624931 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:20.624937 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 11:36:20.625162 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:20.625169 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.626326 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.626329 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:20.627095 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:20.627106 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.627773 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.627776 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:20.627854 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:20.627912 (+ 58us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 11:36:20.628316 (+ 404us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 11:36:20.628323 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 11:36:20.628385 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 11:36:20.628469 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":152,"threads_started":1}
I20260806 11:36:20.628669 23928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:20.624159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:55227 (local address 127.21.14.194:33067)
0806 11:36:20.624326 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:20.624332 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:20.624602 (+ 270us) server_negotiation.cc:407] Connection header received
0806 11:36:20.624736 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:20.624740 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:20.624804 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:20.624908 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 11:36:20.625285 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.626183 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.627235 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.627618 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.627736 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:20.628098 (+ 362us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 11:36:20.628205 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 11:36:20.628298 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:20.628464 (+ 166us) server_negotiation.cc:299] Negotiation successful
0806 11:36:20.628529 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 11:36:20.637634 23844 raft_consensus.cc:1261] T 280a576396d9494a8efc2c431e8f5abf P 66ca4b1903f6452fa22ce688b579d7eb [term 1 FOLLOWER]: Refusing update from remote peer c22a8b491fc74a76a11e016052b574ab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 11:36:20.637882 23705 raft_consensus.cc:1261] T 280a576396d9494a8efc2c431e8f5abf P f62920d5c6274e8987acdec21c17ab85 [term 1 FOLLOWER]: Refusing update from remote peer c22a8b491fc74a76a11e016052b574ab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 11:36:20.638217 23942 consensus_queue.cc:1035] T 280a576396d9494a8efc2c431e8f5abf P c22a8b491fc74a76a11e016052b574ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "66ca4b1903f6452fa22ce688b579d7eb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 43505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 11:36:20.638451 23938 consensus_queue.cc:1035] T 280a576396d9494a8efc2c431e8f5abf P c22a8b491fc74a76a11e016052b574ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "f62920d5c6274e8987acdec21c17ab85" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 33067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 11:36:20.667282 23979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:20.663301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.193:42373 (local address 127.21.14.194:48463)
0806 11:36:20.663724 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:20.663736 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:20.663911 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:20.664181 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:20.664185 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:20.664193 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 11:36:20.664324 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:20.664331 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.665329 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.665333 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:20.665964 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:20.665973 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.666474 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.666478 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:20.666580 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:20.666631 (+ 51us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 11:36:20.666973 (+ 342us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 11:36:20.666982 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 11:36:20.667047 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 11:36:20.667132 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":121,"threads_started":1}
I20260806 11:36:20.667268 23930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:20.663384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:48463 (local address 127.21.14.193:42373)
0806 11:36:20.663551 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:20.663559 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:20.663820 (+ 261us) server_negotiation.cc:407] Connection header received
0806 11:36:20.664032 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:20.664035 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:20.664091 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:20.664170 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 11:36:20.664460 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.665226 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.666088 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.666344 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.666468 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:20.666769 (+ 301us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 11:36:20.666895 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 11:36:20.666965 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:20.667056 (+ 91us) server_negotiation.cc:299] Negotiation successful
0806 11:36:20.667115 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 11:36:20.699741 23402 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39146:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 11:36:20.699986 23402 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39146:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 11:36:20.703220 23402 catalog_manager.cc:5782] T 00000000000000000000000000000000 P de08b384a69445a79d55b1cef258a14e: Sending DeleteTablet for 3 replicas of tablet 280a576396d9494a8efc2c431e8f5abf
I20260806 11:36:20.704319 23838 tablet_service.cc:1514] Processing DeleteTablet for tablet 280a576396d9494a8efc2c431e8f5abf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:36:20 UTC) from {username='kudu', principal='kudu/127.21.14.254@KRBTEST.COM'} at 127.0.0.1:45708
I20260806 11:36:20.704319 23694 tablet_service.cc:1514] Processing DeleteTablet for tablet 280a576396d9494a8efc2c431e8f5abf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:36:20 UTC) from {username='kudu', principal='kudu/127.21.14.254@KRBTEST.COM'} at 127.0.0.1:36828
I20260806 11:36:20.705072 23982 tablet_replica.cc:330] T 280a576396d9494a8efc2c431e8f5abf P f62920d5c6274e8987acdec21c17ab85: stopping tablet replica
I20260806 11:36:20.705296 23548 tablet_service.cc:1514] Processing DeleteTablet for tablet 280a576396d9494a8efc2c431e8f5abf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:36:20 UTC) from {username='kudu', principal='kudu/127.21.14.254@KRBTEST.COM'} at 127.0.0.1:42562
I20260806 11:36:20.705490 23982 raft_consensus.cc:2229] T 280a576396d9494a8efc2c431e8f5abf P f62920d5c6274e8987acdec21c17ab85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 11:36:20.705695 23982 raft_consensus.cc:2258] T 280a576396d9494a8efc2c431e8f5abf P f62920d5c6274e8987acdec21c17ab85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 11:36:20.706075 23983 tablet_replica.cc:330] T 280a576396d9494a8efc2c431e8f5abf P 66ca4b1903f6452fa22ce688b579d7eb: stopping tablet replica
I20260806 11:36:20.706413 23983 raft_consensus.cc:2229] T 280a576396d9494a8efc2c431e8f5abf P 66ca4b1903f6452fa22ce688b579d7eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 11:36:20.706633 23983 raft_consensus.cc:2258] T 280a576396d9494a8efc2c431e8f5abf P 66ca4b1903f6452fa22ce688b579d7eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 11:36:20.707149 23402 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39146:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 11:36:20.707440 23982 ts_tablet_manager.cc:1905] T 280a576396d9494a8efc2c431e8f5abf P f62920d5c6274e8987acdec21c17ab85: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 11:36:20.707669 23402 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 11:36:20.708287 23983 ts_tablet_manager.cc:1905] T 280a576396d9494a8efc2c431e8f5abf P 66ca4b1903f6452fa22ce688b579d7eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 11:36:20.710992 23982 ts_tablet_manager.cc:1918] T 280a576396d9494a8efc2c431e8f5abf P f62920d5c6274e8987acdec21c17ab85: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 11:36:20.711131 23982 log.cc:1196] T 280a576396d9494a8efc2c431e8f5abf P f62920d5c6274e8987acdec21c17ab85: Deleting WAL directory at /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/ts-1/wal/wals/280a576396d9494a8efc2c431e8f5abf
I20260806 11:36:20.711521 23982 ts_tablet_manager.cc:1939] T 280a576396d9494a8efc2c431e8f5abf P f62920d5c6274e8987acdec21c17ab85: Deleting consensus metadata
I20260806 11:36:20.711827 23983 ts_tablet_manager.cc:1918] T 280a576396d9494a8efc2c431e8f5abf P 66ca4b1903f6452fa22ce688b579d7eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 11:36:20.711949 23983 log.cc:1196] T 280a576396d9494a8efc2c431e8f5abf P 66ca4b1903f6452fa22ce688b579d7eb: Deleting WAL directory at /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/ts-2/wal/wals/280a576396d9494a8efc2c431e8f5abf
I20260806 11:36:20.712275 23983 ts_tablet_manager.cc:1939] T 280a576396d9494a8efc2c431e8f5abf P 66ca4b1903f6452fa22ce688b579d7eb: Deleting consensus metadata
I20260806 11:36:20.713665 23388 catalog_manager.cc:4841] TS f62920d5c6274e8987acdec21c17ab85 (127.21.14.194:33067): tablet 280a576396d9494a8efc2c431e8f5abf (table test-table [id=13626845af6d47d5923a862bfad15c79]) successfully deleted
I20260806 11:36:20.713651 23389 catalog_manager.cc:4841] TS 66ca4b1903f6452fa22ce688b579d7eb (127.21.14.195:43505): tablet 280a576396d9494a8efc2c431e8f5abf (table test-table [id=13626845af6d47d5923a862bfad15c79]) successfully deleted
I20260806 11:36:20.715034 23838 tablet_service.cc:1467] Processing CreateTablet for tablet 8b3cd859ef1446ee817667565243be55 (DEFAULT_TABLE table=test-table [id=403281999a3b489898e17574dd969bf8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:36:20.715337 23838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b3cd859ef1446ee817667565243be55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:20.718165 23931 tablet_bootstrap.cc:492] T 8b3cd859ef1446ee817667565243be55 P 66ca4b1903f6452fa22ce688b579d7eb: Bootstrap starting.
I20260806 11:36:20.718717 23548 tablet_service.cc:1467] Processing CreateTablet for tablet 8b3cd859ef1446ee817667565243be55 (DEFAULT_TABLE table=test-table [id=403281999a3b489898e17574dd969bf8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:36:20.719048 23548 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b3cd859ef1446ee817667565243be55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:20.719496 23931 tablet_bootstrap.cc:654] T 8b3cd859ef1446ee817667565243be55 P 66ca4b1903f6452fa22ce688b579d7eb: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:20.721655 23931 tablet_bootstrap.cc:492] T 8b3cd859ef1446ee817667565243be55 P 66ca4b1903f6452fa22ce688b579d7eb: No bootstrap required, opened a new log
I20260806 11:36:20.721773 23931 ts_tablet_manager.cc:1397] T 8b3cd859ef1446ee817667565243be55 P 66ca4b1903f6452fa22ce688b579d7eb: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260806 11:36:20.721866 23694 tablet_service.cc:1467] Processing CreateTablet for tablet 8b3cd859ef1446ee817667565243be55 (DEFAULT_TABLE table=test-table [id=403281999a3b489898e17574dd969bf8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:36:20.722159 23694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b3cd859ef1446ee817667565243be55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:20.722301 23931 raft_consensus.cc:348] T 8b3cd859ef1446ee817667565243be55 P 66ca4b1903f6452fa22ce688b579d7eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c22a8b491fc74a76a11e016052b574ab" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42373 } } peers { permanent_uuid: "f62920d5c6274e8987acdec21c17ab85" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 33067 } } peers { permanent_uuid: "66ca4b1903f6452fa22ce688b579d7eb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 43505 } }
I20260806 11:36:20.722463 23931 raft_consensus.cc:374] T 8b3cd859ef1446ee817667565243be55 P 66ca4b1903f6452fa22ce688b579d7eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:20.722502 23931 raft_consensus.cc:729] T 8b3cd859ef1446ee817667565243be55 P 66ca4b1903f6452fa22ce688b579d7eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66ca4b1903f6452fa22ce688b579d7eb, State: Initialized, Role: FOLLOWER
I20260806 11:36:20.722668 23931 consensus_queue.cc:260] T 8b3cd859ef1446ee817667565243be55 P 66ca4b1903f6452fa22ce688b579d7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c22a8b491fc74a76a11e016052b574ab" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42373 } } peers { permanent_uuid: "f62920d5c6274e8987acdec21c17ab85" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 33067 } } peers { permanent_uuid: "66ca4b1903f6452fa22ce688b579d7eb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 43505 } }
I20260806 11:36:20.723038 23931 ts_tablet_manager.cc:1428] T 8b3cd859ef1446ee817667565243be55 P 66ca4b1903f6452fa22ce688b579d7eb: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 11:36:20.724922 23934 tablet_bootstrap.cc:492] T 8b3cd859ef1446ee817667565243be55 P f62920d5c6274e8987acdec21c17ab85: Bootstrap starting.
I20260806 11:36:20.726050 23934 tablet_bootstrap.cc:654] T 8b3cd859ef1446ee817667565243be55 P f62920d5c6274e8987acdec21c17ab85: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:20.727716 23932 tablet_bootstrap.cc:492] T 8b3cd859ef1446ee817667565243be55 P c22a8b491fc74a76a11e016052b574ab: Bootstrap starting.
I20260806 11:36:20.727865 23937 raft_consensus.cc:482] T 8b3cd859ef1446ee817667565243be55 P 66ca4b1903f6452fa22ce688b579d7eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 11:36:20.727974 23937 raft_consensus.cc:504] T 8b3cd859ef1446ee817667565243be55 P 66ca4b1903f6452fa22ce688b579d7eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c22a8b491fc74a76a11e016052b574ab" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42373 } } peers { permanent_uuid: "f62920d5c6274e8987acdec21c17ab85" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 33067 } } peers { permanent_uuid: "66ca4b1903f6452fa22ce688b579d7eb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 43505 } }
I20260806 11:36:20.728824 23932 tablet_bootstrap.cc:654] T 8b3cd859ef1446ee817667565243be55 P c22a8b491fc74a76a11e016052b574ab: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:20.729298 23937 leader_election.cc:288] T 8b3cd859ef1446ee817667565243be55 P 66ca4b1903f6452fa22ce688b579d7eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c22a8b491fc74a76a11e016052b574ab (127.21.14.193:42373), f62920d5c6274e8987acdec21c17ab85 (127.21.14.194:33067)
I20260806 11:36:20.735076 23797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:20.729644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.193:42373 (local address 127.21.14.195:48631)
0806 11:36:20.729776 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:20.729795 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:20.729897 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:20.730513 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:20.730517 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:20.730536 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:36:20.730653 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:20.730660 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.731741 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.731745 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:20.733034 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:20.733046 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.734589 (+ 1543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.734593 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:20.734716 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:20.734740 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 11:36:20.734811 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 11:36:20.734925 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260806 11:36:20.735854 23932 tablet_bootstrap.cc:492] T 8b3cd859ef1446ee817667565243be55 P c22a8b491fc74a76a11e016052b574ab: No bootstrap required, opened a new log
I20260806 11:36:20.735991 23932 ts_tablet_manager.cc:1397] T 8b3cd859ef1446ee817667565243be55 P c22a8b491fc74a76a11e016052b574ab: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20260806 11:36:20.736195 23934 tablet_bootstrap.cc:492] T 8b3cd859ef1446ee817667565243be55 P f62920d5c6274e8987acdec21c17ab85: No bootstrap required, opened a new log
I20260806 11:36:20.736286 23934 ts_tablet_manager.cc:1397] T 8b3cd859ef1446ee817667565243be55 P f62920d5c6274e8987acdec21c17ab85: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.002s
I20260806 11:36:20.736547 23932 raft_consensus.cc:348] T 8b3cd859ef1446ee817667565243be55 P c22a8b491fc74a76a11e016052b574ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c22a8b491fc74a76a11e016052b574ab" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42373 } } peers { permanent_uuid: "f62920d5c6274e8987acdec21c17ab85" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 33067 } } peers { permanent_uuid: "66ca4b1903f6452fa22ce688b579d7eb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 43505 } }
I20260806 11:36:20.736680 23932 raft_consensus.cc:374] T 8b3cd859ef1446ee817667565243be55 P c22a8b491fc74a76a11e016052b574ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:20.736716 23932 raft_consensus.cc:729] T 8b3cd859ef1446ee817667565243be55 P c22a8b491fc74a76a11e016052b574ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c22a8b491fc74a76a11e016052b574ab, State: Initialized, Role: FOLLOWER
I20260806 11:36:20.736768 23934 raft_consensus.cc:348] T 8b3cd859ef1446ee817667565243be55 P f62920d5c6274e8987acdec21c17ab85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c22a8b491fc74a76a11e016052b574ab" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42373 } } peers { permanent_uuid: "f62920d5c6274e8987acdec21c17ab85" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 33067 } } peers { permanent_uuid: "66ca4b1903f6452fa22ce688b579d7eb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 43505 } }
I20260806 11:36:20.736888 23934 raft_consensus.cc:374] T 8b3cd859ef1446ee817667565243be55 P f62920d5c6274e8987acdec21c17ab85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:20.736920 23934 raft_consensus.cc:729] T 8b3cd859ef1446ee817667565243be55 P f62920d5c6274e8987acdec21c17ab85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f62920d5c6274e8987acdec21c17ab85, State: Initialized, Role: FOLLOWER
I20260806 11:36:20.736937 23930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:20.729745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:48631 (local address 127.21.14.193:42373)
0806 11:36:20.730191 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:20.730196 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:20.730210 (+ 14us) server_negotiation.cc:407] Connection header received
0806 11:36:20.730262 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:20.730265 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:20.730309 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:20.730382 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:36:20.730781 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.731595 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.733714 (+ 2119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.734335 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.734456 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:20.734523 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:20.736712 (+ 2189us) server_negotiation.cc:299] Negotiation successful
0806 11:36:20.736809 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364}
I20260806 11:36:20.736879 23932 consensus_queue.cc:260] T 8b3cd859ef1446ee817667565243be55 P c22a8b491fc74a76a11e016052b574ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c22a8b491fc74a76a11e016052b574ab" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42373 } } peers { permanent_uuid: "f62920d5c6274e8987acdec21c17ab85" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 33067 } } peers { permanent_uuid: "66ca4b1903f6452fa22ce688b579d7eb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 43505 } }
I20260806 11:36:20.737044 23934 consensus_queue.cc:260] T 8b3cd859ef1446ee817667565243be55 P f62920d5c6274e8987acdec21c17ab85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c22a8b491fc74a76a11e016052b574ab" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42373 } } peers { permanent_uuid: "f62920d5c6274e8987acdec21c17ab85" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 33067 } } peers { permanent_uuid: "66ca4b1903f6452fa22ce688b579d7eb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 43505 } }
I20260806 11:36:20.737251 23932 ts_tablet_manager.cc:1428] T 8b3cd859ef1446ee817667565243be55 P c22a8b491fc74a76a11e016052b574ab: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 11:36:20.737304 23934 ts_tablet_manager.cc:1428] T 8b3cd859ef1446ee817667565243be55 P f62920d5c6274e8987acdec21c17ab85: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 11:36:20.737404 23984 tablet_replica.cc:330] T 280a576396d9494a8efc2c431e8f5abf P c22a8b491fc74a76a11e016052b574ab: stopping tablet replica
I20260806 11:36:20.737779 23984 raft_consensus.cc:2229] T 280a576396d9494a8efc2c431e8f5abf P c22a8b491fc74a76a11e016052b574ab [term 1 LEADER]: Raft consensus shutting down.
I20260806 11:36:20.737846 23569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b3cd859ef1446ee817667565243be55" candidate_uuid: "66ca4b1903f6452fa22ce688b579d7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c22a8b491fc74a76a11e016052b574ab" is_pre_election: true
I20260806 11:36:20.737977 23569 raft_consensus.cc:2454] T 8b3cd859ef1446ee817667565243be55 P c22a8b491fc74a76a11e016052b574ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66ca4b1903f6452fa22ce688b579d7eb in term 0.
I20260806 11:36:20.738175 23984 raft_consensus.cc:2258] T 280a576396d9494a8efc2c431e8f5abf P c22a8b491fc74a76a11e016052b574ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 11:36:20.739583 23791 leader_election.cc:302] T 8b3cd859ef1446ee817667565243be55 P 66ca4b1903f6452fa22ce688b579d7eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66ca4b1903f6452fa22ce688b579d7eb, c22a8b491fc74a76a11e016052b574ab; no voters:
I20260806 11:36:20.739929 23937 raft_consensus.cc:2781] T 8b3cd859ef1446ee817667565243be55 P 66ca4b1903f6452fa22ce688b579d7eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 11:36:20.739990 23937 raft_consensus.cc:482] T 8b3cd859ef1446ee817667565243be55 P 66ca4b1903f6452fa22ce688b579d7eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 11:36:20.740031 23937 raft_consensus.cc:3037] T 8b3cd859ef1446ee817667565243be55 P 66ca4b1903f6452fa22ce688b579d7eb [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:36:20.740775 23984 ts_tablet_manager.cc:1905] T 280a576396d9494a8efc2c431e8f5abf P c22a8b491fc74a76a11e016052b574ab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 11:36:20.741336 23937 raft_consensus.cc:504] T 8b3cd859ef1446ee817667565243be55 P 66ca4b1903f6452fa22ce688b579d7eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c22a8b491fc74a76a11e016052b574ab" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42373 } } peers { permanent_uuid: "f62920d5c6274e8987acdec21c17ab85" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 33067 } } peers { permanent_uuid: "66ca4b1903f6452fa22ce688b579d7eb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 43505 } }
I20260806 11:36:20.741775 23937 leader_election.cc:288] T 8b3cd859ef1446ee817667565243be55 P 66ca4b1903f6452fa22ce688b579d7eb [CANDIDATE]: Term 1 election: Requested vote from peers c22a8b491fc74a76a11e016052b574ab (127.21.14.193:42373), f62920d5c6274e8987acdec21c17ab85 (127.21.14.194:33067)
I20260806 11:36:20.742811 23569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b3cd859ef1446ee817667565243be55" candidate_uuid: "66ca4b1903f6452fa22ce688b579d7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c22a8b491fc74a76a11e016052b574ab"
I20260806 11:36:20.742969 23569 raft_consensus.cc:3037] T 8b3cd859ef1446ee817667565243be55 P c22a8b491fc74a76a11e016052b574ab [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:36:20.744158 23569 raft_consensus.cc:2454] T 8b3cd859ef1446ee817667565243be55 P c22a8b491fc74a76a11e016052b574ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66ca4b1903f6452fa22ce688b579d7eb in term 1.
I20260806 11:36:20.744863 23791 leader_election.cc:302] T 8b3cd859ef1446ee817667565243be55 P 66ca4b1903f6452fa22ce688b579d7eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66ca4b1903f6452fa22ce688b579d7eb, c22a8b491fc74a76a11e016052b574ab; no voters:
I20260806 11:36:20.745150 23937 raft_consensus.cc:2781] T 8b3cd859ef1446ee817667565243be55 P 66ca4b1903f6452fa22ce688b579d7eb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:36:20.745378 23937 raft_consensus.cc:686] T 8b3cd859ef1446ee817667565243be55 P 66ca4b1903f6452fa22ce688b579d7eb [term 1 LEADER]: Becoming Leader. State: Replica: 66ca4b1903f6452fa22ce688b579d7eb, State: Running, Role: LEADER
I20260806 11:36:20.745645 23937 consensus_queue.cc:237] T 8b3cd859ef1446ee817667565243be55 P 66ca4b1903f6452fa22ce688b579d7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c22a8b491fc74a76a11e016052b574ab" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42373 } } peers { permanent_uuid: "f62920d5c6274e8987acdec21c17ab85" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 33067 } } peers { permanent_uuid: "66ca4b1903f6452fa22ce688b579d7eb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 43505 } }
I20260806 11:36:20.747493 23984 ts_tablet_manager.cc:1918] T 280a576396d9494a8efc2c431e8f5abf P c22a8b491fc74a76a11e016052b574ab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 11:36:20.747638 23984 log.cc:1196] T 280a576396d9494a8efc2c431e8f5abf P c22a8b491fc74a76a11e016052b574ab: Deleting WAL directory at /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/ts-0/wal/wals/280a576396d9494a8efc2c431e8f5abf
I20260806 11:36:20.748667 23401 catalog_manager.cc:5495] T 8b3cd859ef1446ee817667565243be55 P 66ca4b1903f6452fa22ce688b579d7eb reported cstate change: term changed from 0 to 1, leader changed from <none> to 66ca4b1903f6452fa22ce688b579d7eb (127.21.14.195). New cstate: current_term: 1 leader_uuid: "66ca4b1903f6452fa22ce688b579d7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c22a8b491fc74a76a11e016052b574ab" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f62920d5c6274e8987acdec21c17ab85" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 33067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66ca4b1903f6452fa22ce688b579d7eb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 43505 } health_report { overall_health: HEALTHY } } }
I20260806 11:36:20.749059 23984 ts_tablet_manager.cc:1939] T 280a576396d9494a8efc2c431e8f5abf P c22a8b491fc74a76a11e016052b574ab: Deleting consensus metadata
I20260806 11:36:20.752220 23388 catalog_manager.cc:4841] TS c22a8b491fc74a76a11e016052b574ab (127.21.14.193:42373): tablet 280a576396d9494a8efc2c431e8f5abf (table test-table [id=13626845af6d47d5923a862bfad15c79]) successfully deleted
I20260806 11:36:20.752851 23928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:20.731945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:48789 (local address 127.21.14.194:33067)
0806 11:36:20.732083 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:20.732087 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:20.732337 (+ 250us) server_negotiation.cc:407] Connection header received
0806 11:36:20.732562 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:20.732565 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:20.732621 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:20.732673 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:36:20.733121 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.733854 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.751237 (+ 17383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.751911 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.752021 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:20.752080 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:20.752593 (+ 513us) server_negotiation.cc:299] Negotiation successful
0806 11:36:20.752723 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 11:36:20.752883 23989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:20.729891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.194:33067 (local address 127.21.14.195:48789)
0806 11:36:20.732248 (+ 2357us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:20.732263 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:20.732410 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:20.732825 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:20.732827 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:20.732861 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:36:20.732976 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:20.732983 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.734009 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.734012 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:20.750728 (+ 16716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:20.750754 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.752219 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.752223 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:20.752430 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:20.752455 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 11:36:20.752549 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 11:36:20.752648 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2289,"thread_start_us":92,"threads_started":1}
I20260806 11:36:20.753859 23705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b3cd859ef1446ee817667565243be55" candidate_uuid: "66ca4b1903f6452fa22ce688b579d7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f62920d5c6274e8987acdec21c17ab85" is_pre_election: true
I20260806 11:36:20.754045 23705 raft_consensus.cc:2454] T 8b3cd859ef1446ee817667565243be55 P f62920d5c6274e8987acdec21c17ab85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66ca4b1903f6452fa22ce688b579d7eb in term 0.
I20260806 11:36:20.753865 23714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b3cd859ef1446ee817667565243be55" candidate_uuid: "66ca4b1903f6452fa22ce688b579d7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f62920d5c6274e8987acdec21c17ab85"
I20260806 11:36:20.754405 23714 raft_consensus.cc:3037] T 8b3cd859ef1446ee817667565243be55 P f62920d5c6274e8987acdec21c17ab85 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:36:20.756621 23714 raft_consensus.cc:2454] T 8b3cd859ef1446ee817667565243be55 P f62920d5c6274e8987acdec21c17ab85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66ca4b1903f6452fa22ce688b579d7eb in term 1.
I20260806 11:36:20.770232 23927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:20.765506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45716 (local address 127.21.14.195:43505)
0806 11:36:20.765657 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:20.765662 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:20.765676 (+ 14us) server_negotiation.cc:407] Connection header received
0806 11:36:20.765772 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:20.765775 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:20.765838 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:20.765951 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 11:36:20.766264 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.767105 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.768318 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.768694 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.768798 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:20.769112 (+ 314us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 11:36:20.769724 (+ 612us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 11:36:20.769839 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:20.769930 (+ 91us) server_negotiation.cc:299] Negotiation successful
0806 11:36:20.770012 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 11:36:20.783618 23928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:20.779613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:60179 (local address 127.21.14.194:33067)
0806 11:36:20.779743 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:20.779747 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:20.779885 (+ 138us) server_negotiation.cc:407] Connection header received
0806 11:36:20.779971 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:20.779974 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:20.780026 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:20.780129 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 11:36:20.780429 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.780970 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.782232 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.782487 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.782589 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:20.783076 (+ 487us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 11:36:20.783203 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 11:36:20.783281 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:20.783405 (+ 124us) server_negotiation.cc:299] Negotiation successful
0806 11:36:20.783486 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 11:36:20.783641 23913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:20.779479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.194:33067 (local address 127.21.14.195:60179)
0806 11:36:20.779751 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:20.779766 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:20.779885 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:20.780143 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:20.780147 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:20.780155 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 11:36:20.780289 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:20.780295 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.781082 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.781086 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:20.782063 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:20.782075 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.782634 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.782639 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:20.782750 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:20.782883 (+ 133us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 11:36:20.783299 (+ 416us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 11:36:20.783310 (+ 11us) client_negotiation.cc:769] Sending connection context
0806 11:36:20.783385 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 11:36:20.783488 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150}
I20260806 11:36:20.791221 23569 raft_consensus.cc:1261] T 8b3cd859ef1446ee817667565243be55 P c22a8b491fc74a76a11e016052b574ab [term 1 FOLLOWER]: Refusing update from remote peer 66ca4b1903f6452fa22ce688b579d7eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 11:36:20.791215 23714 raft_consensus.cc:1261] T 8b3cd859ef1446ee817667565243be55 P f62920d5c6274e8987acdec21c17ab85 [term 1 FOLLOWER]: Refusing update from remote peer 66ca4b1903f6452fa22ce688b579d7eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 11:36:20.791779 23937 consensus_queue.cc:1035] T 8b3cd859ef1446ee817667565243be55 P 66ca4b1903f6452fa22ce688b579d7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "f62920d5c6274e8987acdec21c17ab85" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 33067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 11:36:20.792188 23937 consensus_queue.cc:1035] T 8b3cd859ef1446ee817667565243be55 P 66ca4b1903f6452fa22ce688b579d7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c22a8b491fc74a76a11e016052b574ab" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 11:36:20.819175 23979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:20.814536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.195:43505 (local address 127.21.14.194:35421)
0806 11:36:20.814677 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:20.814695 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:20.814788 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:20.815262 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:20.815266 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:20.815274 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 11:36:20.815382 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:20.815388 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.816468 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.816472 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:20.817442 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:20.817454 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.818372 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.818376 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:20.818468 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:20.818502 (+ 34us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 11:36:20.818818 (+ 316us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 11:36:20.818825 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 11:36:20.818912 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 11:36:20.819025 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 11:36:20.819653 23927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:20.814793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:35421 (local address 127.21.14.195:43505)
0806 11:36:20.814949 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:20.814954 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:20.814966 (+ 12us) server_negotiation.cc:407] Connection header received
0806 11:36:20.815023 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:20.815026 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:20.815079 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:20.815147 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 11:36:20.815504 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.816311 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.817825 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:20.818187 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:20.818285 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:20.818634 (+ 349us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 11:36:20.818718 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 11:36:20.818780 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:20.819460 (+ 680us) server_negotiation.cc:299] Negotiation successful
0806 11:36:20.819525 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260806 11:36:20.820338 23962 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 11:36:22.413368 23400 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39146:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 11:36:22.413587 23400 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39146:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 11:36:22.416456 23400 catalog_manager.cc:5782] T 00000000000000000000000000000000 P de08b384a69445a79d55b1cef258a14e: Sending DeleteTablet for 3 replicas of tablet 8b3cd859ef1446ee817667565243be55
I20260806 11:36:22.417204 23548 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b3cd859ef1446ee817667565243be55 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:36:22 UTC) from {username='kudu', principal='kudu/127.21.14.254@KRBTEST.COM'} at 127.0.0.1:42562
I20260806 11:36:22.417227 23838 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b3cd859ef1446ee817667565243be55 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:36:22 UTC) from {username='kudu', principal='kudu/127.21.14.254@KRBTEST.COM'} at 127.0.0.1:45708
I20260806 11:36:22.417263 23693 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b3cd859ef1446ee817667565243be55 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:36:22 UTC) from {username='kudu', principal='kudu/127.21.14.254@KRBTEST.COM'} at 127.0.0.1:36828
I20260806 11:36:22.417650 24023 tablet_replica.cc:330] T 8b3cd859ef1446ee817667565243be55 P 66ca4b1903f6452fa22ce688b579d7eb: stopping tablet replica
I20260806 11:36:22.417860 24023 raft_consensus.cc:2229] T 8b3cd859ef1446ee817667565243be55 P 66ca4b1903f6452fa22ce688b579d7eb [term 1 LEADER]: Raft consensus shutting down.
I20260806 11:36:22.417889 24024 tablet_replica.cc:330] T 8b3cd859ef1446ee817667565243be55 P f62920d5c6274e8987acdec21c17ab85: stopping tablet replica
I20260806 11:36:22.418005 24024 raft_consensus.cc:2229] T 8b3cd859ef1446ee817667565243be55 P f62920d5c6274e8987acdec21c17ab85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 11:36:22.418126 24024 raft_consensus.cc:2258] T 8b3cd859ef1446ee817667565243be55 P f62920d5c6274e8987acdec21c17ab85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 11:36:22.418282 24025 tablet_replica.cc:330] T 8b3cd859ef1446ee817667565243be55 P c22a8b491fc74a76a11e016052b574ab: stopping tablet replica
I20260806 11:36:22.418429 24025 raft_consensus.cc:2229] T 8b3cd859ef1446ee817667565243be55 P c22a8b491fc74a76a11e016052b574ab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 11:36:22.418457 24023 raft_consensus.cc:2258] T 8b3cd859ef1446ee817667565243be55 P 66ca4b1903f6452fa22ce688b579d7eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 11:36:22.418742 24025 raft_consensus.cc:2258] T 8b3cd859ef1446ee817667565243be55 P c22a8b491fc74a76a11e016052b574ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 11:36:22.419409 24023 ts_tablet_manager.cc:1905] T 8b3cd859ef1446ee817667565243be55 P 66ca4b1903f6452fa22ce688b579d7eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 11:36:22.422123 24024 ts_tablet_manager.cc:1905] T 8b3cd859ef1446ee817667565243be55 P f62920d5c6274e8987acdec21c17ab85: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 11:36:22.422942 24023 ts_tablet_manager.cc:1918] T 8b3cd859ef1446ee817667565243be55 P 66ca4b1903f6452fa22ce688b579d7eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 11:36:22.423032 24023 log.cc:1196] T 8b3cd859ef1446ee817667565243be55 P 66ca4b1903f6452fa22ce688b579d7eb: Deleting WAL directory at /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/ts-2/wal/wals/8b3cd859ef1446ee817667565243be55
I20260806 11:36:22.423345 24023 ts_tablet_manager.cc:1939] T 8b3cd859ef1446ee817667565243be55 P 66ca4b1903f6452fa22ce688b579d7eb: Deleting consensus metadata
I20260806 11:36:22.424399 23389 catalog_manager.cc:4841] TS 66ca4b1903f6452fa22ce688b579d7eb (127.21.14.195:43505): tablet 8b3cd859ef1446ee817667565243be55 (table test-table [id=403281999a3b489898e17574dd969bf8]) successfully deleted
I20260806 11:36:22.424863 24025 ts_tablet_manager.cc:1905] T 8b3cd859ef1446ee817667565243be55 P c22a8b491fc74a76a11e016052b574ab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 11:36:22.425493 24024 ts_tablet_manager.cc:1918] T 8b3cd859ef1446ee817667565243be55 P f62920d5c6274e8987acdec21c17ab85: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 11:36:22.425578 24024 log.cc:1196] T 8b3cd859ef1446ee817667565243be55 P f62920d5c6274e8987acdec21c17ab85: Deleting WAL directory at /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/ts-1/wal/wals/8b3cd859ef1446ee817667565243be55
I20260806 11:36:22.425843 24024 ts_tablet_manager.cc:1939] T 8b3cd859ef1446ee817667565243be55 P f62920d5c6274e8987acdec21c17ab85: Deleting consensus metadata
I20260806 11:36:22.426797 23388 catalog_manager.cc:4841] TS f62920d5c6274e8987acdec21c17ab85 (127.21.14.194:33067): tablet 8b3cd859ef1446ee817667565243be55 (table test-table [id=403281999a3b489898e17574dd969bf8]) successfully deleted
I20260806 11:36:22.427802 24025 ts_tablet_manager.cc:1918] T 8b3cd859ef1446ee817667565243be55 P c22a8b491fc74a76a11e016052b574ab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 11:36:22.427913 24025 log.cc:1196] T 8b3cd859ef1446ee817667565243be55 P c22a8b491fc74a76a11e016052b574ab: Deleting WAL directory at /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/ts-0/wal/wals/8b3cd859ef1446ee817667565243be55
I20260806 11:36:22.428159 24025 ts_tablet_manager.cc:1939] T 8b3cd859ef1446ee817667565243be55 P c22a8b491fc74a76a11e016052b574ab: Deleting consensus metadata
I20260806 11:36:22.428932 23388 catalog_manager.cc:4841] TS c22a8b491fc74a76a11e016052b574ab (127.21.14.193:42373): tablet 8b3cd859ef1446ee817667565243be55 (table test-table [id=403281999a3b489898e17574dd969bf8]) successfully deleted
Aug 06 11:36:22 dist-test-slave-7c35 krb5kdc[23356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016182, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 11:36:22 dist-test-slave-7c35 krb5kdc[23356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016182, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:36:22.458396 24033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:22.442391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39162 (local address 127.21.14.254:40863)
0806 11:36:22.442690 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:22.442701 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:22.442783 (+ 82us) server_negotiation.cc:407] Connection header received
0806 11:36:22.443004 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:22.443007 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:22.443055 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:22.443153 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:22.443917 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:22.444514 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:22.445446 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:22.445696 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:22.445810 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:22.454319 (+ 8509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:22.454323 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:22.454325 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:22.454395 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:22.456224 (+ 1829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:22.456796 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:22.456800 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:22.456801 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:22.456897 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:22.457257 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:22.457261 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:22.457263 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:22.457553 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:22.457705 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:22.458014 (+ 309us) server_negotiation.cc:299] Negotiation successful
0806 11:36:22.458172 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":104,"threads_started":1}
I20260806 11:36:22.462101 23400 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39162:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 11:36:22.462766 23400 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 11:36:22.468979 23548 tablet_service.cc:1467] Processing CreateTablet for tablet ae50e40a03c04ba384dce59ef25ba53e (DEFAULT_TABLE table=test-table [id=3e55744f11f1469490b07e47594c8bab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:36:22.469061 23693 tablet_service.cc:1467] Processing CreateTablet for tablet ae50e40a03c04ba384dce59ef25ba53e (DEFAULT_TABLE table=test-table [id=3e55744f11f1469490b07e47594c8bab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:36:22.469142 23838 tablet_service.cc:1467] Processing CreateTablet for tablet ae50e40a03c04ba384dce59ef25ba53e (DEFAULT_TABLE table=test-table [id=3e55744f11f1469490b07e47594c8bab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:36:22.469367 23693 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae50e40a03c04ba384dce59ef25ba53e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:22.469367 23548 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae50e40a03c04ba384dce59ef25ba53e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:22.469494 23838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae50e40a03c04ba384dce59ef25ba53e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:22.472394 24035 tablet_bootstrap.cc:492] T ae50e40a03c04ba384dce59ef25ba53e P f62920d5c6274e8987acdec21c17ab85: Bootstrap starting.
I20260806 11:36:22.472750 24036 tablet_bootstrap.cc:492] T ae50e40a03c04ba384dce59ef25ba53e P 66ca4b1903f6452fa22ce688b579d7eb: Bootstrap starting.
I20260806 11:36:22.473621 24035 tablet_bootstrap.cc:654] T ae50e40a03c04ba384dce59ef25ba53e P f62920d5c6274e8987acdec21c17ab85: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:22.473706 24036 tablet_bootstrap.cc:654] T ae50e40a03c04ba384dce59ef25ba53e P 66ca4b1903f6452fa22ce688b579d7eb: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:22.474468 24034 tablet_bootstrap.cc:492] T ae50e40a03c04ba384dce59ef25ba53e P c22a8b491fc74a76a11e016052b574ab: Bootstrap starting.
I20260806 11:36:22.475240 24035 tablet_bootstrap.cc:492] T ae50e40a03c04ba384dce59ef25ba53e P f62920d5c6274e8987acdec21c17ab85: No bootstrap required, opened a new log
I20260806 11:36:22.475425 24035 ts_tablet_manager.cc:1397] T ae50e40a03c04ba384dce59ef25ba53e P f62920d5c6274e8987acdec21c17ab85: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.001s
I20260806 11:36:22.475608 24034 tablet_bootstrap.cc:654] T ae50e40a03c04ba384dce59ef25ba53e P c22a8b491fc74a76a11e016052b574ab: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:22.475639 24036 tablet_bootstrap.cc:492] T ae50e40a03c04ba384dce59ef25ba53e P 66ca4b1903f6452fa22ce688b579d7eb: No bootstrap required, opened a new log
I20260806 11:36:22.475764 24036 ts_tablet_manager.cc:1397] T ae50e40a03c04ba384dce59ef25ba53e P 66ca4b1903f6452fa22ce688b579d7eb: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 11:36:22.476235 24035 raft_consensus.cc:348] T ae50e40a03c04ba384dce59ef25ba53e P f62920d5c6274e8987acdec21c17ab85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c22a8b491fc74a76a11e016052b574ab" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42373 } } peers { permanent_uuid: "f62920d5c6274e8987acdec21c17ab85" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 33067 } } peers { permanent_uuid: "66ca4b1903f6452fa22ce688b579d7eb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 43505 } }
I20260806 11:36:22.476234 24036 raft_consensus.cc:348] T ae50e40a03c04ba384dce59ef25ba53e P 66ca4b1903f6452fa22ce688b579d7eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c22a8b491fc74a76a11e016052b574ab" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42373 } } peers { permanent_uuid: "f62920d5c6274e8987acdec21c17ab85" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 33067 } } peers { permanent_uuid: "66ca4b1903f6452fa22ce688b579d7eb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 43505 } }
I20260806 11:36:22.476383 24035 raft_consensus.cc:374] T ae50e40a03c04ba384dce59ef25ba53e P f62920d5c6274e8987acdec21c17ab85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:22.476387 24036 raft_consensus.cc:374] T ae50e40a03c04ba384dce59ef25ba53e P 66ca4b1903f6452fa22ce688b579d7eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:22.476425 24035 raft_consensus.cc:729] T ae50e40a03c04ba384dce59ef25ba53e P f62920d5c6274e8987acdec21c17ab85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f62920d5c6274e8987acdec21c17ab85, State: Initialized, Role: FOLLOWER
I20260806 11:36:22.476426 24036 raft_consensus.cc:729] T ae50e40a03c04ba384dce59ef25ba53e P 66ca4b1903f6452fa22ce688b579d7eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66ca4b1903f6452fa22ce688b579d7eb, State: Initialized, Role: FOLLOWER
I20260806 11:36:22.476569 24035 consensus_queue.cc:260] T ae50e40a03c04ba384dce59ef25ba53e P f62920d5c6274e8987acdec21c17ab85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c22a8b491fc74a76a11e016052b574ab" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42373 } } peers { permanent_uuid: "f62920d5c6274e8987acdec21c17ab85" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 33067 } } peers { permanent_uuid: "66ca4b1903f6452fa22ce688b579d7eb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 43505 } }
I20260806 11:36:22.476647 24036 consensus_queue.cc:260] T ae50e40a03c04ba384dce59ef25ba53e P 66ca4b1903f6452fa22ce688b579d7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c22a8b491fc74a76a11e016052b574ab" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42373 } } peers { permanent_uuid: "f62920d5c6274e8987acdec21c17ab85" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 33067 } } peers { permanent_uuid: "66ca4b1903f6452fa22ce688b579d7eb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 43505 } }
I20260806 11:36:22.476893 24035 ts_tablet_manager.cc:1428] T ae50e40a03c04ba384dce59ef25ba53e P f62920d5c6274e8987acdec21c17ab85: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 11:36:22.477092 24034 tablet_bootstrap.cc:492] T ae50e40a03c04ba384dce59ef25ba53e P c22a8b491fc74a76a11e016052b574ab: No bootstrap required, opened a new log
I20260806 11:36:22.477093 24036 ts_tablet_manager.cc:1428] T ae50e40a03c04ba384dce59ef25ba53e P 66ca4b1903f6452fa22ce688b579d7eb: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 11:36:22.477165 24034 ts_tablet_manager.cc:1397] T ae50e40a03c04ba384dce59ef25ba53e P c22a8b491fc74a76a11e016052b574ab: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 11:36:22.477669 24034 raft_consensus.cc:348] T ae50e40a03c04ba384dce59ef25ba53e P c22a8b491fc74a76a11e016052b574ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c22a8b491fc74a76a11e016052b574ab" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42373 } } peers { permanent_uuid: "f62920d5c6274e8987acdec21c17ab85" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 33067 } } peers { permanent_uuid: "66ca4b1903f6452fa22ce688b579d7eb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 43505 } }
I20260806 11:36:22.477802 24034 raft_consensus.cc:374] T ae50e40a03c04ba384dce59ef25ba53e P c22a8b491fc74a76a11e016052b574ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:22.477836 24034 raft_consensus.cc:729] T ae50e40a03c04ba384dce59ef25ba53e P c22a8b491fc74a76a11e016052b574ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c22a8b491fc74a76a11e016052b574ab, State: Initialized, Role: FOLLOWER
I20260806 11:36:22.477967 24034 consensus_queue.cc:260] T ae50e40a03c04ba384dce59ef25ba53e P c22a8b491fc74a76a11e016052b574ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c22a8b491fc74a76a11e016052b574ab" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42373 } } peers { permanent_uuid: "f62920d5c6274e8987acdec21c17ab85" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 33067 } } peers { permanent_uuid: "66ca4b1903f6452fa22ce688b579d7eb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 43505 } }
I20260806 11:36:22.478461 24034 ts_tablet_manager.cc:1428] T ae50e40a03c04ba384dce59ef25ba53e P c22a8b491fc74a76a11e016052b574ab: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260806 11:36:22.495077 24017 raft_consensus.cc:482] T ae50e40a03c04ba384dce59ef25ba53e P f62920d5c6274e8987acdec21c17ab85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 11:36:22.495257 24017 raft_consensus.cc:504] T ae50e40a03c04ba384dce59ef25ba53e P f62920d5c6274e8987acdec21c17ab85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c22a8b491fc74a76a11e016052b574ab" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42373 } } peers { permanent_uuid: "f62920d5c6274e8987acdec21c17ab85" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 33067 } } peers { permanent_uuid: "66ca4b1903f6452fa22ce688b579d7eb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 43505 } }
I20260806 11:36:22.495786 24017 leader_election.cc:288] T ae50e40a03c04ba384dce59ef25ba53e P f62920d5c6274e8987acdec21c17ab85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c22a8b491fc74a76a11e016052b574ab (127.21.14.193:42373), 66ca4b1903f6452fa22ce688b579d7eb (127.21.14.195:43505)
I20260806 11:36:22.496320 23844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae50e40a03c04ba384dce59ef25ba53e" candidate_uuid: "f62920d5c6274e8987acdec21c17ab85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66ca4b1903f6452fa22ce688b579d7eb" is_pre_election: true
I20260806 11:36:22.496307 23568 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae50e40a03c04ba384dce59ef25ba53e" candidate_uuid: "f62920d5c6274e8987acdec21c17ab85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c22a8b491fc74a76a11e016052b574ab" is_pre_election: true
I20260806 11:36:22.496496 23844 raft_consensus.cc:2454] T ae50e40a03c04ba384dce59ef25ba53e P 66ca4b1903f6452fa22ce688b579d7eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f62920d5c6274e8987acdec21c17ab85 in term 0.
I20260806 11:36:22.496523 23568 raft_consensus.cc:2454] T ae50e40a03c04ba384dce59ef25ba53e P c22a8b491fc74a76a11e016052b574ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f62920d5c6274e8987acdec21c17ab85 in term 0.
I20260806 11:36:22.496894 23648 leader_election.cc:302] T ae50e40a03c04ba384dce59ef25ba53e P f62920d5c6274e8987acdec21c17ab85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66ca4b1903f6452fa22ce688b579d7eb, f62920d5c6274e8987acdec21c17ab85; no voters:
I20260806 11:36:22.497102 24017 raft_consensus.cc:2781] T ae50e40a03c04ba384dce59ef25ba53e P f62920d5c6274e8987acdec21c17ab85 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 11:36:22.497175 24017 raft_consensus.cc:482] T ae50e40a03c04ba384dce59ef25ba53e P f62920d5c6274e8987acdec21c17ab85 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 11:36:22.497216 24017 raft_consensus.cc:3037] T ae50e40a03c04ba384dce59ef25ba53e P f62920d5c6274e8987acdec21c17ab85 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:36:22.498323 24017 raft_consensus.cc:504] T ae50e40a03c04ba384dce59ef25ba53e P f62920d5c6274e8987acdec21c17ab85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c22a8b491fc74a76a11e016052b574ab" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42373 } } peers { permanent_uuid: "f62920d5c6274e8987acdec21c17ab85" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 33067 } } peers { permanent_uuid: "66ca4b1903f6452fa22ce688b579d7eb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 43505 } }
I20260806 11:36:22.498853 24017 leader_election.cc:288] T ae50e40a03c04ba384dce59ef25ba53e P f62920d5c6274e8987acdec21c17ab85 [CANDIDATE]: Term 1 election: Requested vote from peers c22a8b491fc74a76a11e016052b574ab (127.21.14.193:42373), 66ca4b1903f6452fa22ce688b579d7eb (127.21.14.195:43505)
I20260806 11:36:22.499315 23844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae50e40a03c04ba384dce59ef25ba53e" candidate_uuid: "f62920d5c6274e8987acdec21c17ab85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66ca4b1903f6452fa22ce688b579d7eb"
I20260806 11:36:22.499334 23568 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae50e40a03c04ba384dce59ef25ba53e" candidate_uuid: "f62920d5c6274e8987acdec21c17ab85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c22a8b491fc74a76a11e016052b574ab"
I20260806 11:36:22.499445 23844 raft_consensus.cc:3037] T ae50e40a03c04ba384dce59ef25ba53e P 66ca4b1903f6452fa22ce688b579d7eb [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:36:22.499528 23568 raft_consensus.cc:3037] T ae50e40a03c04ba384dce59ef25ba53e P c22a8b491fc74a76a11e016052b574ab [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:36:22.500638 23568 raft_consensus.cc:2454] T ae50e40a03c04ba384dce59ef25ba53e P c22a8b491fc74a76a11e016052b574ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f62920d5c6274e8987acdec21c17ab85 in term 1.
I20260806 11:36:22.500732 23844 raft_consensus.cc:2454] T ae50e40a03c04ba384dce59ef25ba53e P 66ca4b1903f6452fa22ce688b579d7eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f62920d5c6274e8987acdec21c17ab85 in term 1.
I20260806 11:36:22.501066 23647 leader_election.cc:302] T ae50e40a03c04ba384dce59ef25ba53e P f62920d5c6274e8987acdec21c17ab85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c22a8b491fc74a76a11e016052b574ab, f62920d5c6274e8987acdec21c17ab85; no voters:
I20260806 11:36:22.501349 24017 raft_consensus.cc:2781] T ae50e40a03c04ba384dce59ef25ba53e P f62920d5c6274e8987acdec21c17ab85 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:36:22.501459 24017 raft_consensus.cc:686] T ae50e40a03c04ba384dce59ef25ba53e P f62920d5c6274e8987acdec21c17ab85 [term 1 LEADER]: Becoming Leader. State: Replica: f62920d5c6274e8987acdec21c17ab85, State: Running, Role: LEADER
I20260806 11:36:22.501623 24017 consensus_queue.cc:237] T ae50e40a03c04ba384dce59ef25ba53e P f62920d5c6274e8987acdec21c17ab85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c22a8b491fc74a76a11e016052b574ab" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42373 } } peers { permanent_uuid: "f62920d5c6274e8987acdec21c17ab85" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 33067 } } peers { permanent_uuid: "66ca4b1903f6452fa22ce688b579d7eb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 43505 } }
I20260806 11:36:22.503580 23401 catalog_manager.cc:5495] T ae50e40a03c04ba384dce59ef25ba53e P f62920d5c6274e8987acdec21c17ab85 reported cstate change: term changed from 0 to 1, leader changed from <none> to f62920d5c6274e8987acdec21c17ab85 (127.21.14.194). New cstate: current_term: 1 leader_uuid: "f62920d5c6274e8987acdec21c17ab85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c22a8b491fc74a76a11e016052b574ab" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f62920d5c6274e8987acdec21c17ab85" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 33067 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "66ca4b1903f6452fa22ce688b579d7eb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 43505 } health_report { overall_health: UNKNOWN } } }
I20260806 11:36:22.527658 24043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:22.523697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36848 (local address 127.21.14.194:33067)
0806 11:36:22.523990 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:22.523999 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:22.524012 (+ 13us) server_negotiation.cc:407] Connection header received
0806 11:36:22.524066 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:22.524068 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:22.524113 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:22.524190 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 11:36:22.524506 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:22.525139 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:22.526041 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:22.526311 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:22.526581 (+ 270us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:22.526927 (+ 346us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 11:36:22.527176 (+ 249us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 11:36:22.527255 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:22.527421 (+ 166us) server_negotiation.cc:299] Negotiation successful
0806 11:36:22.527482 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":135,"threads_started":1}
I20260806 11:36:22.538014 24043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:22.534837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:38901 (local address 127.21.14.194:33067)
0806 11:36:22.534995 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:22.534999 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:22.535013 (+ 14us) server_negotiation.cc:407] Connection header received
0806 11:36:22.535147 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:22.535150 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:22.535197 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:22.535272 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 11:36:22.535540 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:22.536068 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:22.536931 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:22.537149 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:22.537259 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:22.537521 (+ 262us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 11:36:22.537597 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 11:36:22.537674 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:22.537768 (+ 94us) server_negotiation.cc:299] Negotiation successful
0806 11:36:22.537835 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":34,"server-negotiator.queue_time_us":90}
I20260806 11:36:22.538014 24047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:22.534527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.194:33067 (local address 127.21.14.194:38901)
0806 11:36:22.534841 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:22.534870 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:22.535021 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:22.535292 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:22.535295 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:22.535301 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 11:36:22.535421 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:22.535429 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:22.536178 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:22.536183 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:22.536805 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:22.536814 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:22.537264 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:22.537267 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:22.537367 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:22.537391 (+ 24us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 11:36:22.537693 (+ 302us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 11:36:22.537700 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 11:36:22.537761 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 11:36:22.537852 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"mutex_wait_us":2,"thread_start_us":142,"threads_started":1}
I20260806 11:36:22.545393 23568 raft_consensus.cc:1261] T ae50e40a03c04ba384dce59ef25ba53e P c22a8b491fc74a76a11e016052b574ab [term 1 FOLLOWER]: Refusing update from remote peer f62920d5c6274e8987acdec21c17ab85: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 11:36:22.545480 23844 raft_consensus.cc:1261] T ae50e40a03c04ba384dce59ef25ba53e P 66ca4b1903f6452fa22ce688b579d7eb [term 1 FOLLOWER]: Refusing update from remote peer f62920d5c6274e8987acdec21c17ab85: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 11:36:22.545886 23939 consensus_queue.cc:1035] T ae50e40a03c04ba384dce59ef25ba53e P f62920d5c6274e8987acdec21c17ab85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c22a8b491fc74a76a11e016052b574ab" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 11:36:22.546125 24007 consensus_queue.cc:1035] T ae50e40a03c04ba384dce59ef25ba53e P f62920d5c6274e8987acdec21c17ab85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66ca4b1903f6452fa22ce688b579d7eb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 43505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 11:36:22.587683 23401 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39162:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 11:36:22.587889 23401 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39162:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 11:36:22.590551 23401 catalog_manager.cc:5782] T 00000000000000000000000000000000 P de08b384a69445a79d55b1cef258a14e: Sending DeleteTablet for 3 replicas of tablet ae50e40a03c04ba384dce59ef25ba53e
I20260806 11:36:22.591404 23548 tablet_service.cc:1514] Processing DeleteTablet for tablet ae50e40a03c04ba384dce59ef25ba53e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:36:22 UTC) from {username='kudu', principal='kudu/127.21.14.254@KRBTEST.COM'} at 127.0.0.1:42562
I20260806 11:36:22.591408 23693 tablet_service.cc:1514] Processing DeleteTablet for tablet ae50e40a03c04ba384dce59ef25ba53e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:36:22 UTC) from {username='kudu', principal='kudu/127.21.14.254@KRBTEST.COM'} at 127.0.0.1:36828
I20260806 11:36:22.591645 24025 tablet_replica.cc:330] T ae50e40a03c04ba384dce59ef25ba53e P c22a8b491fc74a76a11e016052b574ab: stopping tablet replica
I20260806 11:36:22.591791 24025 raft_consensus.cc:2229] T ae50e40a03c04ba384dce59ef25ba53e P c22a8b491fc74a76a11e016052b574ab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 11:36:22.591806 23838 tablet_service.cc:1514] Processing DeleteTablet for tablet ae50e40a03c04ba384dce59ef25ba53e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:36:22 UTC) from {username='kudu', principal='kudu/127.21.14.254@KRBTEST.COM'} at 127.0.0.1:45708
I20260806 11:36:22.591944 24025 raft_consensus.cc:2258] T ae50e40a03c04ba384dce59ef25ba53e P c22a8b491fc74a76a11e016052b574ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 11:36:22.592047 24024 tablet_replica.cc:330] T ae50e40a03c04ba384dce59ef25ba53e P f62920d5c6274e8987acdec21c17ab85: stopping tablet replica
I20260806 11:36:22.592177 24023 tablet_replica.cc:330] T ae50e40a03c04ba384dce59ef25ba53e P 66ca4b1903f6452fa22ce688b579d7eb: stopping tablet replica
I20260806 11:36:22.592203 24024 raft_consensus.cc:2229] T ae50e40a03c04ba384dce59ef25ba53e P f62920d5c6274e8987acdec21c17ab85 [term 1 LEADER]: Raft consensus shutting down.
I20260806 11:36:22.592315 24023 raft_consensus.cc:2229] T ae50e40a03c04ba384dce59ef25ba53e P 66ca4b1903f6452fa22ce688b579d7eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 11:36:22.592448 24023 raft_consensus.cc:2258] T ae50e40a03c04ba384dce59ef25ba53e P 66ca4b1903f6452fa22ce688b579d7eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 11:36:22.592608 24024 raft_consensus.cc:2258] T ae50e40a03c04ba384dce59ef25ba53e P f62920d5c6274e8987acdec21c17ab85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 11:36:22.593276 24023 ts_tablet_manager.cc:1905] T ae50e40a03c04ba384dce59ef25ba53e P 66ca4b1903f6452fa22ce688b579d7eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 11:36:22.593367 24024 ts_tablet_manager.cc:1905] T ae50e40a03c04ba384dce59ef25ba53e P f62920d5c6274e8987acdec21c17ab85: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 11:36:22.593621 24025 ts_tablet_manager.cc:1905] T ae50e40a03c04ba384dce59ef25ba53e P c22a8b491fc74a76a11e016052b574ab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 11:36:22.596486 24023 ts_tablet_manager.cc:1918] T ae50e40a03c04ba384dce59ef25ba53e P 66ca4b1903f6452fa22ce688b579d7eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 11:36:22.596544 24025 ts_tablet_manager.cc:1918] T ae50e40a03c04ba384dce59ef25ba53e P c22a8b491fc74a76a11e016052b574ab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 11:36:22.596643 24025 log.cc:1196] T ae50e40a03c04ba384dce59ef25ba53e P c22a8b491fc74a76a11e016052b574ab: Deleting WAL directory at /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/ts-0/wal/wals/ae50e40a03c04ba384dce59ef25ba53e
I20260806 11:36:22.596799 24023 log.cc:1196] T ae50e40a03c04ba384dce59ef25ba53e P 66ca4b1903f6452fa22ce688b579d7eb: Deleting WAL directory at /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/ts-2/wal/wals/ae50e40a03c04ba384dce59ef25ba53e
I20260806 11:36:22.596917 24025 ts_tablet_manager.cc:1939] T ae50e40a03c04ba384dce59ef25ba53e P c22a8b491fc74a76a11e016052b574ab: Deleting consensus metadata
I20260806 11:36:22.597302 24024 ts_tablet_manager.cc:1918] T ae50e40a03c04ba384dce59ef25ba53e P f62920d5c6274e8987acdec21c17ab85: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 11:36:22.597391 24024 log.cc:1196] T ae50e40a03c04ba384dce59ef25ba53e P f62920d5c6274e8987acdec21c17ab85: Deleting WAL directory at /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/ts-1/wal/wals/ae50e40a03c04ba384dce59ef25ba53e
I20260806 11:36:22.597419 23401 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39162:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 11:36:22.597677 24024 ts_tablet_manager.cc:1939] T ae50e40a03c04ba384dce59ef25ba53e P f62920d5c6274e8987acdec21c17ab85: Deleting consensus metadata
I20260806 11:36:22.597932 23388 catalog_manager.cc:4841] TS c22a8b491fc74a76a11e016052b574ab (127.21.14.193:42373): tablet ae50e40a03c04ba384dce59ef25ba53e (table test-table [id=3e55744f11f1469490b07e47594c8bab]) successfully deleted
W20260806 11:36:22.598061 23401 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 11:36:22.598644 23388 catalog_manager.cc:4841] TS f62920d5c6274e8987acdec21c17ab85 (127.21.14.194:33067): tablet ae50e40a03c04ba384dce59ef25ba53e (table test-table [id=3e55744f11f1469490b07e47594c8bab]) successfully deleted
I20260806 11:36:22.599346 24023 ts_tablet_manager.cc:1939] T ae50e40a03c04ba384dce59ef25ba53e P 66ca4b1903f6452fa22ce688b579d7eb: Deleting consensus metadata
I20260806 11:36:22.600090 23389 catalog_manager.cc:4841] TS 66ca4b1903f6452fa22ce688b579d7eb (127.21.14.195:43505): tablet ae50e40a03c04ba384dce59ef25ba53e (table test-table [id=3e55744f11f1469490b07e47594c8bab]) successfully deleted
I20260806 11:36:22.603785 23548 tablet_service.cc:1467] Processing CreateTablet for tablet 2301e7c50d1342c1a51e2c8bdf22b269 (DEFAULT_TABLE table=test-table [id=9c7a90de50f54c53ac54afd2634e389e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:36:22.603833 23693 tablet_service.cc:1467] Processing CreateTablet for tablet 2301e7c50d1342c1a51e2c8bdf22b269 (DEFAULT_TABLE table=test-table [id=9c7a90de50f54c53ac54afd2634e389e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:36:22.604133 23548 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2301e7c50d1342c1a51e2c8bdf22b269. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:22.604423 23838 tablet_service.cc:1467] Processing CreateTablet for tablet 2301e7c50d1342c1a51e2c8bdf22b269 (DEFAULT_TABLE table=test-table [id=9c7a90de50f54c53ac54afd2634e389e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:36:22.604724 23838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2301e7c50d1342c1a51e2c8bdf22b269. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:22.606761 24034 tablet_bootstrap.cc:492] T 2301e7c50d1342c1a51e2c8bdf22b269 P c22a8b491fc74a76a11e016052b574ab: Bootstrap starting.
I20260806 11:36:22.606997 24036 tablet_bootstrap.cc:492] T 2301e7c50d1342c1a51e2c8bdf22b269 P 66ca4b1903f6452fa22ce688b579d7eb: Bootstrap starting.
I20260806 11:36:22.607904 24036 tablet_bootstrap.cc:654] T 2301e7c50d1342c1a51e2c8bdf22b269 P 66ca4b1903f6452fa22ce688b579d7eb: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:22.607978 24034 tablet_bootstrap.cc:654] T 2301e7c50d1342c1a51e2c8bdf22b269 P c22a8b491fc74a76a11e016052b574ab: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:22.609499 24034 tablet_bootstrap.cc:492] T 2301e7c50d1342c1a51e2c8bdf22b269 P c22a8b491fc74a76a11e016052b574ab: No bootstrap required, opened a new log
I20260806 11:36:22.609498 24036 tablet_bootstrap.cc:492] T 2301e7c50d1342c1a51e2c8bdf22b269 P 66ca4b1903f6452fa22ce688b579d7eb: No bootstrap required, opened a new log
I20260806 11:36:22.609611 24034 ts_tablet_manager.cc:1397] T 2301e7c50d1342c1a51e2c8bdf22b269 P c22a8b491fc74a76a11e016052b574ab: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 11:36:22.609625 24036 ts_tablet_manager.cc:1397] T 2301e7c50d1342c1a51e2c8bdf22b269 P 66ca4b1903f6452fa22ce688b579d7eb: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 11:36:22.610097 24034 raft_consensus.cc:348] T 2301e7c50d1342c1a51e2c8bdf22b269 P c22a8b491fc74a76a11e016052b574ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c22a8b491fc74a76a11e016052b574ab" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42373 } } peers { permanent_uuid: "f62920d5c6274e8987acdec21c17ab85" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 33067 } } peers { permanent_uuid: "66ca4b1903f6452fa22ce688b579d7eb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 43505 } }
I20260806 11:36:22.610097 24036 raft_consensus.cc:348] T 2301e7c50d1342c1a51e2c8bdf22b269 P 66ca4b1903f6452fa22ce688b579d7eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c22a8b491fc74a76a11e016052b574ab" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42373 } } peers { permanent_uuid: "f62920d5c6274e8987acdec21c17ab85" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 33067 } } peers { permanent_uuid: "66ca4b1903f6452fa22ce688b579d7eb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 43505 } }
I20260806 11:36:22.610224 24034 raft_consensus.cc:374] T 2301e7c50d1342c1a51e2c8bdf22b269 P c22a8b491fc74a76a11e016052b574ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:22.610224 24036 raft_consensus.cc:374] T 2301e7c50d1342c1a51e2c8bdf22b269 P 66ca4b1903f6452fa22ce688b579d7eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:22.610258 24034 raft_consensus.cc:729] T 2301e7c50d1342c1a51e2c8bdf22b269 P c22a8b491fc74a76a11e016052b574ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c22a8b491fc74a76a11e016052b574ab, State: Initialized, Role: FOLLOWER
I20260806 11:36:22.610258 24036 raft_consensus.cc:729] T 2301e7c50d1342c1a51e2c8bdf22b269 P 66ca4b1903f6452fa22ce688b579d7eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66ca4b1903f6452fa22ce688b579d7eb, State: Initialized, Role: FOLLOWER
I20260806 11:36:22.610410 24034 consensus_queue.cc:260] T 2301e7c50d1342c1a51e2c8bdf22b269 P c22a8b491fc74a76a11e016052b574ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c22a8b491fc74a76a11e016052b574ab" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42373 } } peers { permanent_uuid: "f62920d5c6274e8987acdec21c17ab85" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 33067 } } peers { permanent_uuid: "66ca4b1903f6452fa22ce688b579d7eb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 43505 } }
I20260806 11:36:22.610733 24034 ts_tablet_manager.cc:1428] T 2301e7c50d1342c1a51e2c8bdf22b269 P c22a8b491fc74a76a11e016052b574ab: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 11:36:22.610918 24036 consensus_queue.cc:260] T 2301e7c50d1342c1a51e2c8bdf22b269 P 66ca4b1903f6452fa22ce688b579d7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c22a8b491fc74a76a11e016052b574ab" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42373 } } peers { permanent_uuid: "f62920d5c6274e8987acdec21c17ab85" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 33067 } } peers { permanent_uuid: "66ca4b1903f6452fa22ce688b579d7eb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 43505 } }
I20260806 11:36:22.611342 24036 ts_tablet_manager.cc:1428] T 2301e7c50d1342c1a51e2c8bdf22b269 P 66ca4b1903f6452fa22ce688b579d7eb: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260806 11:36:22.604175 23693 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2301e7c50d1342c1a51e2c8bdf22b269. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:22.615576 24035 tablet_bootstrap.cc:492] T 2301e7c50d1342c1a51e2c8bdf22b269 P f62920d5c6274e8987acdec21c17ab85: Bootstrap starting.
I20260806 11:36:22.616565 24035 tablet_bootstrap.cc:654] T 2301e7c50d1342c1a51e2c8bdf22b269 P f62920d5c6274e8987acdec21c17ab85: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:22.617937 24035 tablet_bootstrap.cc:492] T 2301e7c50d1342c1a51e2c8bdf22b269 P f62920d5c6274e8987acdec21c17ab85: No bootstrap required, opened a new log
I20260806 11:36:22.618031 24035 ts_tablet_manager.cc:1397] T 2301e7c50d1342c1a51e2c8bdf22b269 P f62920d5c6274e8987acdec21c17ab85: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260806 11:36:22.618534 24035 raft_consensus.cc:348] T 2301e7c50d1342c1a51e2c8bdf22b269 P f62920d5c6274e8987acdec21c17ab85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c22a8b491fc74a76a11e016052b574ab" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42373 } } peers { permanent_uuid: "f62920d5c6274e8987acdec21c17ab85" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 33067 } } peers { permanent_uuid: "66ca4b1903f6452fa22ce688b579d7eb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 43505 } }
I20260806 11:36:22.618644 24035 raft_consensus.cc:374] T 2301e7c50d1342c1a51e2c8bdf22b269 P f62920d5c6274e8987acdec21c17ab85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:22.618666 24035 raft_consensus.cc:729] T 2301e7c50d1342c1a51e2c8bdf22b269 P f62920d5c6274e8987acdec21c17ab85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f62920d5c6274e8987acdec21c17ab85, State: Initialized, Role: FOLLOWER
I20260806 11:36:22.618810 24035 consensus_queue.cc:260] T 2301e7c50d1342c1a51e2c8bdf22b269 P f62920d5c6274e8987acdec21c17ab85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c22a8b491fc74a76a11e016052b574ab" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42373 } } peers { permanent_uuid: "f62920d5c6274e8987acdec21c17ab85" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 33067 } } peers { permanent_uuid: "66ca4b1903f6452fa22ce688b579d7eb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 43505 } }
I20260806 11:36:22.619197 24035 ts_tablet_manager.cc:1428] T 2301e7c50d1342c1a51e2c8bdf22b269 P f62920d5c6274e8987acdec21c17ab85: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260806 11:36:22.682957 24015 raft_consensus.cc:482] T 2301e7c50d1342c1a51e2c8bdf22b269 P 66ca4b1903f6452fa22ce688b579d7eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 11:36:22.683140 24015 raft_consensus.cc:504] T 2301e7c50d1342c1a51e2c8bdf22b269 P 66ca4b1903f6452fa22ce688b579d7eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c22a8b491fc74a76a11e016052b574ab" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42373 } } peers { permanent_uuid: "f62920d5c6274e8987acdec21c17ab85" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 33067 } } peers { permanent_uuid: "66ca4b1903f6452fa22ce688b579d7eb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 43505 } }
I20260806 11:36:22.683681 24015 leader_election.cc:288] T 2301e7c50d1342c1a51e2c8bdf22b269 P 66ca4b1903f6452fa22ce688b579d7eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c22a8b491fc74a76a11e016052b574ab (127.21.14.193:42373), f62920d5c6274e8987acdec21c17ab85 (127.21.14.194:33067)
I20260806 11:36:22.684291 23569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2301e7c50d1342c1a51e2c8bdf22b269" candidate_uuid: "66ca4b1903f6452fa22ce688b579d7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c22a8b491fc74a76a11e016052b574ab" is_pre_election: true
I20260806 11:36:22.684351 23714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2301e7c50d1342c1a51e2c8bdf22b269" candidate_uuid: "66ca4b1903f6452fa22ce688b579d7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f62920d5c6274e8987acdec21c17ab85" is_pre_election: true
I20260806 11:36:22.684477 23569 raft_consensus.cc:2454] T 2301e7c50d1342c1a51e2c8bdf22b269 P c22a8b491fc74a76a11e016052b574ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66ca4b1903f6452fa22ce688b579d7eb in term 0.
I20260806 11:36:22.684517 23714 raft_consensus.cc:2454] T 2301e7c50d1342c1a51e2c8bdf22b269 P f62920d5c6274e8987acdec21c17ab85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66ca4b1903f6452fa22ce688b579d7eb in term 0.
I20260806 11:36:22.684849 23791 leader_election.cc:302] T 2301e7c50d1342c1a51e2c8bdf22b269 P 66ca4b1903f6452fa22ce688b579d7eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66ca4b1903f6452fa22ce688b579d7eb, f62920d5c6274e8987acdec21c17ab85; no voters:
I20260806 11:36:22.685062 24015 raft_consensus.cc:2781] T 2301e7c50d1342c1a51e2c8bdf22b269 P 66ca4b1903f6452fa22ce688b579d7eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 11:36:22.685177 24015 raft_consensus.cc:482] T 2301e7c50d1342c1a51e2c8bdf22b269 P 66ca4b1903f6452fa22ce688b579d7eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 11:36:22.685220 24015 raft_consensus.cc:3037] T 2301e7c50d1342c1a51e2c8bdf22b269 P 66ca4b1903f6452fa22ce688b579d7eb [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:36:22.686635 24015 raft_consensus.cc:504] T 2301e7c50d1342c1a51e2c8bdf22b269 P 66ca4b1903f6452fa22ce688b579d7eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c22a8b491fc74a76a11e016052b574ab" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42373 } } peers { permanent_uuid: "f62920d5c6274e8987acdec21c17ab85" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 33067 } } peers { permanent_uuid: "66ca4b1903f6452fa22ce688b579d7eb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 43505 } }
I20260806 11:36:22.687040 24015 leader_election.cc:288] T 2301e7c50d1342c1a51e2c8bdf22b269 P 66ca4b1903f6452fa22ce688b579d7eb [CANDIDATE]: Term 1 election: Requested vote from peers c22a8b491fc74a76a11e016052b574ab (127.21.14.193:42373), f62920d5c6274e8987acdec21c17ab85 (127.21.14.194:33067)
I20260806 11:36:22.687481 23714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2301e7c50d1342c1a51e2c8bdf22b269" candidate_uuid: "66ca4b1903f6452fa22ce688b579d7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f62920d5c6274e8987acdec21c17ab85"
I20260806 11:36:22.687500 23569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2301e7c50d1342c1a51e2c8bdf22b269" candidate_uuid: "66ca4b1903f6452fa22ce688b579d7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c22a8b491fc74a76a11e016052b574ab"
I20260806 11:36:22.687633 23569 raft_consensus.cc:3037] T 2301e7c50d1342c1a51e2c8bdf22b269 P c22a8b491fc74a76a11e016052b574ab [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:36:22.687660 23714 raft_consensus.cc:3037] T 2301e7c50d1342c1a51e2c8bdf22b269 P f62920d5c6274e8987acdec21c17ab85 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:36:22.688709 23714 raft_consensus.cc:2454] T 2301e7c50d1342c1a51e2c8bdf22b269 P f62920d5c6274e8987acdec21c17ab85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66ca4b1903f6452fa22ce688b579d7eb in term 1.
I20260806 11:36:22.688875 23569 raft_consensus.cc:2454] T 2301e7c50d1342c1a51e2c8bdf22b269 P c22a8b491fc74a76a11e016052b574ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66ca4b1903f6452fa22ce688b579d7eb in term 1.
I20260806 11:36:22.689167 23791 leader_election.cc:302] T 2301e7c50d1342c1a51e2c8bdf22b269 P 66ca4b1903f6452fa22ce688b579d7eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66ca4b1903f6452fa22ce688b579d7eb, f62920d5c6274e8987acdec21c17ab85; no voters:
I20260806 11:36:22.689388 24015 raft_consensus.cc:2781] T 2301e7c50d1342c1a51e2c8bdf22b269 P 66ca4b1903f6452fa22ce688b579d7eb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:36:22.689575 24015 raft_consensus.cc:686] T 2301e7c50d1342c1a51e2c8bdf22b269 P 66ca4b1903f6452fa22ce688b579d7eb [term 1 LEADER]: Becoming Leader. State: Replica: 66ca4b1903f6452fa22ce688b579d7eb, State: Running, Role: LEADER
I20260806 11:36:22.689747 24015 consensus_queue.cc:237] T 2301e7c50d1342c1a51e2c8bdf22b269 P 66ca4b1903f6452fa22ce688b579d7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c22a8b491fc74a76a11e016052b574ab" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42373 } } peers { permanent_uuid: "f62920d5c6274e8987acdec21c17ab85" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 33067 } } peers { permanent_uuid: "66ca4b1903f6452fa22ce688b579d7eb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 43505 } }
I20260806 11:36:22.692286 23401 catalog_manager.cc:5495] T 2301e7c50d1342c1a51e2c8bdf22b269 P 66ca4b1903f6452fa22ce688b579d7eb reported cstate change: term changed from 0 to 1, leader changed from <none> to 66ca4b1903f6452fa22ce688b579d7eb (127.21.14.195). New cstate: current_term: 1 leader_uuid: "66ca4b1903f6452fa22ce688b579d7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c22a8b491fc74a76a11e016052b574ab" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f62920d5c6274e8987acdec21c17ab85" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 33067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66ca4b1903f6452fa22ce688b579d7eb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 43505 } health_report { overall_health: HEALTHY } } }
I20260806 11:36:22.720903 24055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:22.717096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45724 (local address 127.21.14.195:43505)
0806 11:36:22.717379 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:22.717386 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:22.717399 (+ 13us) server_negotiation.cc:407] Connection header received
0806 11:36:22.717445 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:22.717447 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:22.717491 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:22.717596 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 11:36:22.717879 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:22.718416 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:22.719646 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:22.719896 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:22.719983 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:22.720325 (+ 342us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 11:36:22.720405 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 11:36:22.720464 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:22.720632 (+ 168us) server_negotiation.cc:299] Negotiation successful
0806 11:36:22.720688 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":128,"threads_started":1}
I20260806 11:36:22.729360 23569 raft_consensus.cc:1261] T 2301e7c50d1342c1a51e2c8bdf22b269 P c22a8b491fc74a76a11e016052b574ab [term 1 FOLLOWER]: Refusing update from remote peer 66ca4b1903f6452fa22ce688b579d7eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 11:36:22.729360 23714 raft_consensus.cc:1261] T 2301e7c50d1342c1a51e2c8bdf22b269 P f62920d5c6274e8987acdec21c17ab85 [term 1 FOLLOWER]: Refusing update from remote peer 66ca4b1903f6452fa22ce688b579d7eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 11:36:22.729874 24012 consensus_queue.cc:1035] T 2301e7c50d1342c1a51e2c8bdf22b269 P 66ca4b1903f6452fa22ce688b579d7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c22a8b491fc74a76a11e016052b574ab" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 11:36:22.730070 24015 consensus_queue.cc:1035] T 2301e7c50d1342c1a51e2c8bdf22b269 P 66ca4b1903f6452fa22ce688b579d7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "f62920d5c6274e8987acdec21c17ab85" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 33067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 11:36:22.748041 24002 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 11:36:24.315330 23401 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39162:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 11:36:24.315532 23401 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39162:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 11:36:24.318737 23401 catalog_manager.cc:5782] T 00000000000000000000000000000000 P de08b384a69445a79d55b1cef258a14e: Sending DeleteTablet for 3 replicas of tablet 2301e7c50d1342c1a51e2c8bdf22b269
I20260806 11:36:24.319594 23693 tablet_service.cc:1514] Processing DeleteTablet for tablet 2301e7c50d1342c1a51e2c8bdf22b269 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:36:24 UTC) from {username='kudu', principal='kudu/127.21.14.254@KRBTEST.COM'} at 127.0.0.1:36828
I20260806 11:36:24.319636 23548 tablet_service.cc:1514] Processing DeleteTablet for tablet 2301e7c50d1342c1a51e2c8bdf22b269 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:36:24 UTC) from {username='kudu', principal='kudu/127.21.14.254@KRBTEST.COM'} at 127.0.0.1:42562
I20260806 11:36:24.319995 23838 tablet_service.cc:1514] Processing DeleteTablet for tablet 2301e7c50d1342c1a51e2c8bdf22b269 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:36:24 UTC) from {username='kudu', principal='kudu/127.21.14.254@KRBTEST.COM'} at 127.0.0.1:45708
I20260806 11:36:24.320199 24082 tablet_replica.cc:330] T 2301e7c50d1342c1a51e2c8bdf22b269 P f62920d5c6274e8987acdec21c17ab85: stopping tablet replica
I20260806 11:36:24.320369 24085 tablet_replica.cc:330] T 2301e7c50d1342c1a51e2c8bdf22b269 P 66ca4b1903f6452fa22ce688b579d7eb: stopping tablet replica
I20260806 11:36:24.320381 24082 raft_consensus.cc:2229] T 2301e7c50d1342c1a51e2c8bdf22b269 P f62920d5c6274e8987acdec21c17ab85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 11:36:24.320484 24085 raft_consensus.cc:2229] T 2301e7c50d1342c1a51e2c8bdf22b269 P 66ca4b1903f6452fa22ce688b579d7eb [term 1 LEADER]: Raft consensus shutting down.
I20260806 11:36:24.320523 24082 raft_consensus.cc:2258] T 2301e7c50d1342c1a51e2c8bdf22b269 P f62920d5c6274e8987acdec21c17ab85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 11:36:24.320708 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 23481
I20260806 11:36:24.320736 24085 raft_consensus.cc:2258] T 2301e7c50d1342c1a51e2c8bdf22b269 P 66ca4b1903f6452fa22ce688b579d7eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 11:36:24.322152 24085 ts_tablet_manager.cc:1905] T 2301e7c50d1342c1a51e2c8bdf22b269 P 66ca4b1903f6452fa22ce688b579d7eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 11:36:24.328346 24082 ts_tablet_manager.cc:1905] T 2301e7c50d1342c1a51e2c8bdf22b269 P f62920d5c6274e8987acdec21c17ab85: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 11:36:24.332662 23388 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.14.193:42373 (error 108)
I20260806 11:36:24.332772 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 23628
I20260806 11:36:24.335520 24087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:24.334786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.193:42373 (local address 127.0.0.1:42578)
0806 11:36:24.335283 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:24.335409 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.193:42373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":233,"threads_started":1}
W20260806 11:36:24.335693 23388 catalog_manager.cc:4584] TS c22a8b491fc74a76a11e016052b574ab (127.21.14.193:42373): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2301e7c50d1342c1a51e2c8bdf22b269: Network error: Client connection negotiation failed: client connection to 127.21.14.193:42373: connect: Connection refused (error 111)
I20260806 11:36:24.389082 24087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:24.388737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.193:42373 (local address 127.0.0.1:42586)
0806 11:36:24.388902 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:24.388980 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.193:42373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260806 11:36:24.390038 24087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:24.389713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.193:42373 (local address 127.0.0.1:42592)
0806 11:36:24.389870 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:24.389954 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.193:42373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260806 11:36:24.395488 24085 ts_tablet_manager.cc:1918] T 2301e7c50d1342c1a51e2c8bdf22b269 P 66ca4b1903f6452fa22ce688b579d7eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 11:36:24.395618 24085 log.cc:1196] T 2301e7c50d1342c1a51e2c8bdf22b269 P 66ca4b1903f6452fa22ce688b579d7eb: Deleting WAL directory at /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786016169045816-21563-0/minicluster-data/ts-2/wal/wals/2301e7c50d1342c1a51e2c8bdf22b269
I20260806 11:36:24.395972 24085 ts_tablet_manager.cc:1939] T 2301e7c50d1342c1a51e2c8bdf22b269 P 66ca4b1903f6452fa22ce688b579d7eb: Deleting consensus metadata
I20260806 11:36:24.400694 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 23771
I20260806 11:36:24.403522 24087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:24.403069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.194:33067 (local address 127.0.0.1:36852)
0806 11:36:24.403348 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:24.403430 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.194:33067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160}
I20260806 11:36:24.435956 24087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:24.435566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.194:33067 (local address 127.0.0.1:36864)
0806 11:36:24.435769 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:24.435856 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.194:33067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260806 11:36:24.436877 24087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:24.436599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.194:33067 (local address 127.0.0.1:36878)
0806 11:36:24.436724 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:24.436791 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.194:33067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260806 11:36:24.447425 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 23370
2026-08-06T11:36:24Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6498 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 11:36:24 dist-test-slave-7c35 krb5kdc[24090](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 11:36:24 dist-test-slave-7c35 krb5kdc[24090](info): set up 2 sockets
Aug 06 11:36:24 dist-test-slave-7c35 krb5kdc[24090](info): commencing operation
krb5kdc: starting...
W20260806 11:36:25.732046 21563 mini_kdc.cc:121] Time spent starting KDC: real 1.255s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Aug 06 11:36:25 dist-test-slave-7c35 krb5kdc[24090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016185, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T11:36:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T11:36:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.14.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab.
Entry for principal kudu/127.21.14.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab.
I20260806 11:36:25.874298 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:33111
--webserver_interface=127.21.14.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab
--principal=kudu/127.21.14.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:33235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.14.254:33111
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:36:26.015964 24104 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:36:26.016314 24104 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:36:26.016398 24104 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:36:26.020577 24104 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 11:36:26.020691 24104 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:36:26.020730 24104 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:36:26.020771 24104 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 11:36:26.020807 24104 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 11:36:26.026490 24104 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:33235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.14.254:33111
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab
--principal=kudu/127.21.14.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:33111
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.14.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24104
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:36:26.027856 24104 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:36:26.029260 24104 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:36:26.039335 24111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:26.039356 24113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:26.039402 24104 server_base.cc:1034] running on GCE node
W20260806 11:36:26.039335 24110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:26.040390 24104 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:36:26.041613 24104 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:36:26.042815 24104 hybrid_clock.cc:648] HybridClock initialized: now 1786016186042798 us; error 58 us; skew 500 ppm
Aug 06 11:36:26 dist-test-slave-7c35 krb5kdc[24090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016186, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:36:26.045987 24104 init.cc:377] Logged in from keytab as kudu/127.21.14.254@KRBTEST.COM (short username kudu)
I20260806 11:36:26.047370 24104 webserver.cc:460] Webserver started at http://127.21.14.254:43167/ using document root <none> and password file <none>
I20260806 11:36:26.048040 24104 fs_manager.cc:359] Metadata directory not provided
I20260806 11:36:26.048143 24104 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:36:26.048408 24104 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:36:26.050774 24104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/minicluster-data/master-0/data/instance:
uuid: "01c7f8a6605a4b788379ef4bf06fb103"
format_stamp: "Formatted at 2026-08-06 11:36:26 on dist-test-slave-7c35"
server_key: "6ad07cff88313c8d32fa2afa1d17493a"
server_key_iv: "5ba94a209476e0fdb4c592ce9db8e4b2"
server_key_version: "encryptionkey@0"
I20260806 11:36:26.051496 24104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/minicluster-data/master-0/wal/instance:
uuid: "01c7f8a6605a4b788379ef4bf06fb103"
format_stamp: "Formatted at 2026-08-06 11:36:26 on dist-test-slave-7c35"
server_key: "6ad07cff88313c8d32fa2afa1d17493a"
server_key_iv: "5ba94a209476e0fdb4c592ce9db8e4b2"
server_key_version: "encryptionkey@0"
I20260806 11:36:26.055807 24104 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 11:36:26.058588 24121 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:26.060087 24104 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 11:36:26.060271 24104 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/minicluster-data/master-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/minicluster-data/master-0/wal
uuid: "01c7f8a6605a4b788379ef4bf06fb103"
format_stamp: "Formatted at 2026-08-06 11:36:26 on dist-test-slave-7c35"
server_key: "6ad07cff88313c8d32fa2afa1d17493a"
server_key_iv: "5ba94a209476e0fdb4c592ce9db8e4b2"
server_key_version: "encryptionkey@0"
I20260806 11:36:26.060405 24104 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:36:26.074221 24104 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:36:26.077572 24104 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:36:26.077814 24104 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:36:26.087167 24104 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.254:33111
I20260806 11:36:26.087241 24177 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.254:33111 every 8 connection(s)
I20260806 11:36:26.088362 24104 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
I20260806 11:36:26.092067 24178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:26.093113 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 24104
I20260806 11:36:26.093235 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/minicluster-data/master-0/wal/instance
I20260806 11:36:26.093544 21563 external_mini_cluster.cc:1428] Setting key 40fa56d5a21b16a718d000d0373d6310
I20260806 11:36:26.099576 24178 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01c7f8a6605a4b788379ef4bf06fb103: Bootstrap starting.
I20260806 11:36:26.102665 24178 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 01c7f8a6605a4b788379ef4bf06fb103: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:26.103737 24178 log.cc:824] T 00000000000000000000000000000000 P 01c7f8a6605a4b788379ef4bf06fb103: Log is configured to *not* fsync() on all Append() calls
Aug 06 11:36:26 dist-test-slave-7c35 krb5kdc[24090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016185, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:36:26.105950 24178 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01c7f8a6605a4b788379ef4bf06fb103: No bootstrap required, opened a new log
I20260806 11:36:26.109830 24178 raft_consensus.cc:348] T 00000000000000000000000000000000 P 01c7f8a6605a4b788379ef4bf06fb103 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01c7f8a6605a4b788379ef4bf06fb103" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 33111 } }
I20260806 11:36:26.110011 24181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:26.095250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51014 (local address 127.21.14.254:33111)
0806 11:36:26.095802 (+ 552us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:26.095820 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:26.095864 (+ 44us) server_negotiation.cc:407] Connection header received
0806 11:36:26.096705 (+ 841us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:26.096726 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:26.097108 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:26.097418 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:26.098456 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:26.099664 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:26.100911 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:26.101468 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:26.101619 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:26.104793 (+ 3174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:26.104799 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:26.104815 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:26.104854 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:26.106772 (+ 1918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:26.107366 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:26.107372 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:26.107386 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:26.107511 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:26.107821 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:26.107824 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:26.107826 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:26.108277 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:26.108492 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:26.108818 (+ 326us) server_negotiation.cc:299] Negotiation successful
0806 11:36:26.109144 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":195,"threads_started":1}
I20260806 11:36:26.110133 24178 raft_consensus.cc:374] T 00000000000000000000000000000000 P 01c7f8a6605a4b788379ef4bf06fb103 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:26.110204 24178 raft_consensus.cc:729] T 00000000000000000000000000000000 P 01c7f8a6605a4b788379ef4bf06fb103 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01c7f8a6605a4b788379ef4bf06fb103, State: Initialized, Role: FOLLOWER
I20260806 11:36:26.110661 24178 consensus_queue.cc:260] T 00000000000000000000000000000000 P 01c7f8a6605a4b788379ef4bf06fb103 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01c7f8a6605a4b788379ef4bf06fb103" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 33111 } }
I20260806 11:36:26.110836 24178 raft_consensus.cc:388] T 00000000000000000000000000000000 P 01c7f8a6605a4b788379ef4bf06fb103 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 11:36:26.110950 24178 raft_consensus.cc:482] T 00000000000000000000000000000000 P 01c7f8a6605a4b788379ef4bf06fb103 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 11:36:26.111064 24178 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 01c7f8a6605a4b788379ef4bf06fb103 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:36:26.112059 24178 raft_consensus.cc:504] T 00000000000000000000000000000000 P 01c7f8a6605a4b788379ef4bf06fb103 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01c7f8a6605a4b788379ef4bf06fb103" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 33111 } }
I20260806 11:36:26.112453 24178 leader_election.cc:302] T 00000000000000000000000000000000 P 01c7f8a6605a4b788379ef4bf06fb103 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 01c7f8a6605a4b788379ef4bf06fb103; no voters:
I20260806 11:36:26.112809 24178 leader_election.cc:288] T 00000000000000000000000000000000 P 01c7f8a6605a4b788379ef4bf06fb103 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 11:36:26.112988 24183 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 01c7f8a6605a4b788379ef4bf06fb103 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:36:26.113243 24183 raft_consensus.cc:686] T 00000000000000000000000000000000 P 01c7f8a6605a4b788379ef4bf06fb103 [term 1 LEADER]: Becoming Leader. State: Replica: 01c7f8a6605a4b788379ef4bf06fb103, State: Running, Role: LEADER
I20260806 11:36:26.113646 24183 consensus_queue.cc:237] T 00000000000000000000000000000000 P 01c7f8a6605a4b788379ef4bf06fb103 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01c7f8a6605a4b788379ef4bf06fb103" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 33111 } }
I20260806 11:36:26.114146 24178 sys_catalog.cc:564] T 00000000000000000000000000000000 P 01c7f8a6605a4b788379ef4bf06fb103 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 11:36:26.116012 24184 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01c7f8a6605a4b788379ef4bf06fb103 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "01c7f8a6605a4b788379ef4bf06fb103" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01c7f8a6605a4b788379ef4bf06fb103" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 33111 } } }
I20260806 11:36:26.116164 24184 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01c7f8a6605a4b788379ef4bf06fb103 [sys.catalog]: This master's current role is: LEADER
I20260806 11:36:26.116031 24185 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01c7f8a6605a4b788379ef4bf06fb103 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01c7f8a6605a4b788379ef4bf06fb103. Latest consensus state: current_term: 1 leader_uuid: "01c7f8a6605a4b788379ef4bf06fb103" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01c7f8a6605a4b788379ef4bf06fb103" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 33111 } } }
I20260806 11:36:26.116465 24185 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01c7f8a6605a4b788379ef4bf06fb103 [sys.catalog]: This master's current role is: LEADER
W20260806 11:36:26.120251 24198 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 01c7f8a6605a4b788379ef4bf06fb103: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 11:36:26.120385 24198 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 11:36:26.120522 24196 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 11:36:26.121695 24196 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 11:36:26.129236 24196 catalog_manager.cc:1347] Generated new cluster ID: fb6642a0193d49be8ef4d561c50c3590
I20260806 11:36:26.129372 24196 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 11:36:26.144240 24196 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 11:36:26.145427 24196 catalog_manager.cc:1504] Loading token signing keys...
I20260806 11:36:26.151715 24196 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 01c7f8a6605a4b788379ef4bf06fb103: Generated new TSK 0
I20260806 11:36:26.152657 24196 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.14.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab.
Entry for principal kudu/127.21.14.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab.
I20260806 11:36:26.194872 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.193:0
--local_ip_for_outbound_sockets=127.21.14.193
--webserver_interface=127.21.14.193
--webserver_port=0
--tserver_master_addrs=127.21.14.254:33111
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab
--principal=kudu/127.21.14.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:33235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:36:26.326826 24204 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:36:26.327239 24204 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:36:26.327322 24204 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:36:26.331670 24204 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:36:26.331749 24204 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:36:26.331845 24204 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.193
I20260806 11:36:26.337615 24204 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:33235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab
--principal=kudu/127.21.14.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.14.193
--webserver_port=0
--tserver_master_addrs=127.21.14.254:33111
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24204
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.193
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:36:26.338876 24204 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:36:26.340014 24204 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:36:26.350790 24213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:26.350978 24204 server_base.cc:1034] running on GCE node
W20260806 11:36:26.350908 24211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:26.350908 24210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:26.351614 24204 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:36:26.352429 24204 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:36:26.353652 24204 hybrid_clock.cc:648] HybridClock initialized: now 1786016186353605 us; error 60 us; skew 500 ppm
Aug 06 11:36:26 dist-test-slave-7c35 krb5kdc[24090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016186, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:36:26.357412 24204 init.cc:377] Logged in from keytab as kudu/127.21.14.193@KRBTEST.COM (short username kudu)
I20260806 11:36:26.358455 24204 webserver.cc:460] Webserver started at http://127.21.14.193:43745/ using document root <none> and password file <none>
I20260806 11:36:26.359155 24204 fs_manager.cc:359] Metadata directory not provided
I20260806 11:36:26.359231 24204 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:36:26.359423 24204 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:36:26.361728 24204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-0/data/instance:
uuid: "8918bb0159304fdf87c15f3c3d07357c"
format_stamp: "Formatted at 2026-08-06 11:36:26 on dist-test-slave-7c35"
server_key: "71809ddb402cb9ecda7abadfab890ccf"
server_key_iv: "87bbc4089e878f5a77a44699a3d7fac3"
server_key_version: "encryptionkey@0"
I20260806 11:36:26.362305 24204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-0/wal/instance:
uuid: "8918bb0159304fdf87c15f3c3d07357c"
format_stamp: "Formatted at 2026-08-06 11:36:26 on dist-test-slave-7c35"
server_key: "71809ddb402cb9ecda7abadfab890ccf"
server_key_iv: "87bbc4089e878f5a77a44699a3d7fac3"
server_key_version: "encryptionkey@0"
I20260806 11:36:26.367311 24204 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260806 11:36:26.370540 24220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:26.372574 24204 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 11:36:26.372830 24204 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-0/wal
uuid: "8918bb0159304fdf87c15f3c3d07357c"
format_stamp: "Formatted at 2026-08-06 11:36:26 on dist-test-slave-7c35"
server_key: "71809ddb402cb9ecda7abadfab890ccf"
server_key_iv: "87bbc4089e878f5a77a44699a3d7fac3"
server_key_version: "encryptionkey@0"
I20260806 11:36:26.373025 24204 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:36:26.390395 24204 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:36:26.394426 24204 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:36:26.394688 24204 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:36:26.395483 24204 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:36:26.396517 24204 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:36:26.396600 24204 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:26.396678 24204 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:36:26.396723 24204 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:26.406289 24204 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.193:35895
I20260806 11:36:26.406338 24341 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.193:35895 every 8 connection(s)
I20260806 11:36:26.407402 24204 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
I20260806 11:36:26.413869 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 24204
I20260806 11:36:26.414024 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-0/wal/instance
I20260806 11:36:26.414306 21563 external_mini_cluster.cc:1428] Setting key 5baab7f16a0693c6f05090f581a326e5
Aug 06 11:36:26 dist-test-slave-7c35 krb5kdc[24090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016186, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.193@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:36:26.423851 24181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:26.409545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:49553 (local address 127.21.14.254:33111)
0806 11:36:26.409709 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:26.409714 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:26.410921 (+ 1207us) server_negotiation.cc:407] Connection header received
0806 11:36:26.411873 (+ 952us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:26.411876 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:26.411928 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:26.412007 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:26.413294 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:26.414016 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:26.415850 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:26.416153 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:26.416284 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:26.419259 (+ 2975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:26.419266 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:26.419270 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:26.419307 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:26.421078 (+ 1771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:26.421698 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:26.421705 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:26.421709 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:26.421832 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:26.422355 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:26.422360 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:26.422364 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:26.422614 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:26.422746 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:26.423463 (+ 717us) server_negotiation.cc:299] Negotiation successful
0806 11:36:26.423631 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 11:36:26.424755 24344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:26.410015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:33111 (local address 127.21.14.193:49553)
0806 11:36:26.410733 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:26.410778 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:26.411675 (+ 897us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:26.412194 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:26.412201 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:26.412618 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:26.413142 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:26.413158 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:26.414198 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:26.414202 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:26.415633 (+ 1431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:26.415646 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:26.416335 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:26.416340 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:26.416568 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:26.417274 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:26.417292 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:26.419012 (+ 1720us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:26.421221 (+ 2209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:26.421227 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:26.421242 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:26.421537 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:26.421986 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:26.421990 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:26.421991 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:26.422185 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:26.422758 (+ 573us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:26.422769 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:26.423159 (+ 390us) client_negotiation.cc:769] Sending connection context
0806 11:36:26.423397 (+ 238us) client_negotiation.cc:240] Negotiation successful
0806 11:36:26.423785 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":114,"threads_started":1}
WARNING: no policy specified for kudu/127.21.14.194@KRBTEST.COM; defaulting to no policy
I20260806 11:36:26.426303 24342 heartbeater.cc:347] Connected to a master server at 127.21.14.254:33111
I20260806 11:36:26.426637 24342 heartbeater.cc:464] Registering TS with master...
I20260806 11:36:26.427286 24342 heartbeater.cc:511] Master 127.21.14.254:33111 requested a full tablet report, sending...
I20260806 11:36:26.429278 24141 ts_manager.cc:194] Registered new tserver with Master: 8918bb0159304fdf87c15f3c3d07357c (127.21.14.193:35895)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.194@KRBTEST.COM" created.
I20260806 11:36:26.430652 24141 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.14.193@KRBTEST.COM'} at 127.21.14.193:49553
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab.
Entry for principal kudu/127.21.14.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab.
I20260806 11:36:26.451747 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.194:0
--local_ip_for_outbound_sockets=127.21.14.194
--webserver_interface=127.21.14.194
--webserver_port=0
--tserver_master_addrs=127.21.14.254:33111
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab
--principal=kudu/127.21.14.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:33235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:36:26.576300 24347 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:36:26.576622 24347 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:36:26.576747 24347 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:36:26.581064 24347 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:36:26.581133 24347 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:36:26.581195 24347 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.194
I20260806 11:36:26.586663 24347 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:33235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab
--principal=kudu/127.21.14.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.14.194
--webserver_port=0
--tserver_master_addrs=127.21.14.254:33111
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24347
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.194
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:36:26.587909 24347 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:36:26.588979 24347 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:36:26.599766 24356 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:26.599747 24352 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:26.599756 24354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:26.599844 24347 server_base.cc:1034] running on GCE node
I20260806 11:36:26.600605 24347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:36:26.601382 24347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:36:26.602583 24347 hybrid_clock.cc:648] HybridClock initialized: now 1786016186602562 us; error 35 us; skew 500 ppm
Aug 06 11:36:26 dist-test-slave-7c35 krb5kdc[24090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016186, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:36:26.605819 24347 init.cc:377] Logged in from keytab as kudu/127.21.14.194@KRBTEST.COM (short username kudu)
I20260806 11:36:26.607126 24347 webserver.cc:460] Webserver started at http://127.21.14.194:44119/ using document root <none> and password file <none>
I20260806 11:36:26.607803 24347 fs_manager.cc:359] Metadata directory not provided
I20260806 11:36:26.607921 24347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:36:26.608196 24347 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:36:26.610718 24347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-1/data/instance:
uuid: "721fe8ce01a441beb8b606ee6c556b29"
format_stamp: "Formatted at 2026-08-06 11:36:26 on dist-test-slave-7c35"
server_key: "aa27533c4a900fe9b743b5b9802f9c27"
server_key_iv: "1f902f8c867f04643f3bca52818b2d73"
server_key_version: "encryptionkey@0"
I20260806 11:36:26.611672 24347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-1/wal/instance:
uuid: "721fe8ce01a441beb8b606ee6c556b29"
format_stamp: "Formatted at 2026-08-06 11:36:26 on dist-test-slave-7c35"
server_key: "aa27533c4a900fe9b743b5b9802f9c27"
server_key_iv: "1f902f8c867f04643f3bca52818b2d73"
server_key_version: "encryptionkey@0"
I20260806 11:36:26.616619 24347 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 11:36:26.619678 24363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:26.621330 24347 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 11:36:26.621531 24347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-1/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-1/wal
uuid: "721fe8ce01a441beb8b606ee6c556b29"
format_stamp: "Formatted at 2026-08-06 11:36:26 on dist-test-slave-7c35"
server_key: "aa27533c4a900fe9b743b5b9802f9c27"
server_key_iv: "1f902f8c867f04643f3bca52818b2d73"
server_key_version: "encryptionkey@0"
I20260806 11:36:26.621680 24347 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:36:26.646063 24347 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:36:26.649722 24347 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:36:26.649984 24347 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:36:26.650678 24347 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:36:26.651752 24347 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:36:26.651827 24347 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:26.651896 24347 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:36:26.651940 24347 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:26.661420 24347 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.194:33747
I20260806 11:36:26.661446 24489 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.194:33747 every 8 connection(s)
I20260806 11:36:26.662384 24347 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
I20260806 11:36:26.670671 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 24347
I20260806 11:36:26.670790 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-1/wal/instance
I20260806 11:36:26.671141 21563 external_mini_cluster.cc:1428] Setting key 800d791660ba25c39d699f93aa05b60d
Aug 06 11:36:26 dist-test-slave-7c35 krb5kdc[24090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016186, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.194@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:36:26.679831 24181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:26.664482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:59185 (local address 127.21.14.254:33111)
0806 11:36:26.664643 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:26.664647 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:26.665613 (+ 966us) server_negotiation.cc:407] Connection header received
0806 11:36:26.667466 (+ 1853us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:26.667470 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:26.667545 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:26.667654 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:26.669107 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:26.669678 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:26.671320 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:26.671720 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:26.671837 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:26.675179 (+ 3342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:26.675183 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:26.675185 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:26.675214 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:26.676884 (+ 1670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:26.677757 (+ 873us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:26.677761 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:26.677762 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:26.677863 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:26.678365 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:26.678369 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:26.678370 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:26.678629 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:26.678722 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:26.679565 (+ 843us) server_negotiation.cc:299] Negotiation successful
0806 11:36:26.679691 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 11:36:26.681397 24492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:26.664871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:33111 (local address 127.21.14.194:59185)
0806 11:36:26.665425 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:26.665475 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:26.667101 (+ 1626us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:26.667900 (+ 799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:26.667908 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:26.668362 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:26.668909 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:26.668924 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:26.669848 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:26.669851 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:26.671154 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:26.671166 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:26.671948 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:26.671953 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:26.672199 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:26.673002 (+ 803us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:26.673026 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:26.674960 (+ 1934us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:26.677142 (+ 2182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:26.677154 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:26.677178 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:26.677620 (+ 442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:26.677970 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:26.677976 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:26.677981 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:26.678217 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:26.678764 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:26.678775 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:26.679235 (+ 460us) client_negotiation.cc:769] Sending connection context
0806 11:36:26.679492 (+ 257us) client_negotiation.cc:240] Negotiation successful
0806 11:36:26.679933 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":108,"threads_started":1}
I20260806 11:36:26.682966 24490 heartbeater.cc:347] Connected to a master server at 127.21.14.254:33111
I20260806 11:36:26.683274 24490 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.21.14.195@KRBTEST.COM; defaulting to no policy
I20260806 11:36:26.683864 24490 heartbeater.cc:511] Master 127.21.14.254:33111 requested a full tablet report, sending...
I20260806 11:36:26.685245 24141 ts_manager.cc:194] Registered new tserver with Master: 721fe8ce01a441beb8b606ee6c556b29 (127.21.14.194:33747)
I20260806 11:36:26.685928 24141 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.14.194@KRBTEST.COM'} at 127.21.14.194:59185
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab.
Entry for principal kudu/127.21.14.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab.
I20260806 11:36:26.706907 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.195:0
--local_ip_for_outbound_sockets=127.21.14.195
--webserver_interface=127.21.14.195
--webserver_port=0
--tserver_master_addrs=127.21.14.254:33111
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab
--principal=kudu/127.21.14.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:33235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:36:26.831975 24495 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:36:26.832316 24495 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:36:26.832397 24495 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:36:26.837291 24495 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:36:26.837392 24495 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:36:26.837495 24495 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.195
I20260806 11:36:26.843240 24495 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:33235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab
--principal=kudu/127.21.14.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.14.195
--webserver_port=0
--tserver_master_addrs=127.21.14.254:33111
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24495
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.195
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:36:26.844460 24495 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:36:26.845598 24495 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:36:26.856298 24503 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:26.856276 24505 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:26.856253 24502 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:26.856702 24495 server_base.cc:1034] running on GCE node
I20260806 11:36:26.857234 24495 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:36:26.857993 24495 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:36:26.859189 24495 hybrid_clock.cc:648] HybridClock initialized: now 1786016186859139 us; error 66 us; skew 500 ppm
Aug 06 11:36:26 dist-test-slave-7c35 krb5kdc[24090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016186, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:36:26.862196 24495 init.cc:377] Logged in from keytab as kudu/127.21.14.195@KRBTEST.COM (short username kudu)
I20260806 11:36:26.863325 24495 webserver.cc:460] Webserver started at http://127.21.14.195:34981/ using document root <none> and password file <none>
I20260806 11:36:26.863950 24495 fs_manager.cc:359] Metadata directory not provided
I20260806 11:36:26.864053 24495 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:36:26.864357 24495 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:36:26.866597 24495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-2/data/instance:
uuid: "8f51b7f577864b6299e452aadb5fa638"
format_stamp: "Formatted at 2026-08-06 11:36:26 on dist-test-slave-7c35"
server_key: "9623bbe642353bef7312575ca175bdf9"
server_key_iv: "ad9bcb320885c7cc8262aa62420ebffb"
server_key_version: "encryptionkey@0"
I20260806 11:36:26.867331 24495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-2/wal/instance:
uuid: "8f51b7f577864b6299e452aadb5fa638"
format_stamp: "Formatted at 2026-08-06 11:36:26 on dist-test-slave-7c35"
server_key: "9623bbe642353bef7312575ca175bdf9"
server_key_iv: "ad9bcb320885c7cc8262aa62420ebffb"
server_key_version: "encryptionkey@0"
I20260806 11:36:26.871634 24495 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 11:36:26.874505 24513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:26.876116 24495 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 11:36:26.876260 24495 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-2/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-2/wal
uuid: "8f51b7f577864b6299e452aadb5fa638"
format_stamp: "Formatted at 2026-08-06 11:36:26 on dist-test-slave-7c35"
server_key: "9623bbe642353bef7312575ca175bdf9"
server_key_iv: "ad9bcb320885c7cc8262aa62420ebffb"
server_key_version: "encryptionkey@0"
I20260806 11:36:26.876391 24495 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:36:26.896224 24495 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:36:26.900166 24495 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:36:26.900367 24495 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:36:26.900990 24495 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:36:26.901949 24495 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:36:26.901999 24495 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:26.902074 24495 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:36:26.902107 24495 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:26.912442 24495 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.195:38807
I20260806 11:36:26.912451 24638 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.195:38807 every 8 connection(s)
I20260806 11:36:26.913480 24495 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
I20260806 11:36:26.915257 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 24495
I20260806 11:36:26.915387 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-2/wal/instance
I20260806 11:36:26.915752 21563 external_mini_cluster.cc:1428] Setting key bc0991cc681f11c559387d768b5f97d3
Aug 06 11:36:26 dist-test-slave-7c35 krb5kdc[24090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016186, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.195@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:36:26.930228 24181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:26.915622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:36539 (local address 127.21.14.254:33111)
0806 11:36:26.915799 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:26.915806 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:26.917358 (+ 1552us) server_negotiation.cc:407] Connection header received
0806 11:36:26.918597 (+ 1239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:26.918600 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:26.918648 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:26.918717 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:26.920031 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:26.920593 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:26.921880 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:26.922135 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:26.922236 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:26.925353 (+ 3117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:26.925356 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:26.925359 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:26.925390 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:26.927248 (+ 1858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:26.927964 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:26.927968 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:26.927969 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:26.928069 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:26.928572 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:26.928576 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:26.928577 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:26.928896 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:26.929019 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:26.929873 (+ 854us) server_negotiation.cc:299] Negotiation successful
0806 11:36:26.930027 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 11:36:26.931351 24641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:26.916153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:33111 (local address 127.21.14.195:36539)
0806 11:36:26.917200 (+ 1047us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:26.917255 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:26.918362 (+ 1107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:26.918927 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:26.918935 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:26.919320 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:26.919855 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:26.919877 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:26.920731 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:26.920734 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:26.921728 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:26.921739 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:26.922338 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:26.922341 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:26.922521 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:26.923218 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:26.923236 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:26.925120 (+ 1884us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:26.927437 (+ 2317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:26.927445 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:26.927461 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:26.927828 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:26.928175 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:26.928177 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:26.928178 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:26.928422 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:26.929053 (+ 631us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:26.929061 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:26.929506 (+ 445us) client_negotiation.cc:769] Sending connection context
0806 11:36:26.929766 (+ 260us) client_negotiation.cc:240] Negotiation successful
0806 11:36:26.930186 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":792,"thread_start_us":152,"threads_started":1}
I20260806 11:36:26.932799 24639 heartbeater.cc:347] Connected to a master server at 127.21.14.254:33111
I20260806 11:36:26.933100 24639 heartbeater.cc:464] Registering TS with master...
I20260806 11:36:26.933665 24639 heartbeater.cc:511] Master 127.21.14.254:33111 requested a full tablet report, sending...
I20260806 11:36:26.934918 24141 ts_manager.cc:194] Registered new tserver with Master: 8f51b7f577864b6299e452aadb5fa638 (127.21.14.195:38807)
I20260806 11:36:26.935518 24141 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.14.195@KRBTEST.COM'} at 127.21.14.195:36539
I20260806 11:36:26.942800 21563 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 11:36:26.960421 24181 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:36:26.958452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51026 (local address 127.21.14.254:33111)
0806 11:36:26.958648 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:26.958652 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:26.958825 (+ 173us) server_negotiation.cc:407] Connection header received
0806 11:36:26.959055 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:26.959058 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:26.959114 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:26.959304 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:26.960258 (+ 954us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51026: BlockingRecv error: recv got EOF from 127.0.0.1:51026 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
I20260806 11:36:26.961546 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 24204
I20260806 11:36:26.969357 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 24347
I20260806 11:36:26.976351 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 24495
I20260806 11:36:26.985280 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 24104
2026-08-06T11:36:26Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2534 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786016169045816-21563-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 11:36:27 dist-test-slave-7c35 krb5kdc[24654](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 11:36:27 dist-test-slave-7c35 krb5kdc[24654](info): set up 2 sockets
Aug 06 11:36:27 dist-test-slave-7c35 krb5kdc[24654](info): commencing operation
krb5kdc: starting...
W20260806 11:36:28.087263 21563 mini_kdc.cc:121] Time spent starting KDC: real 1.077s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Aug 06 11:36:28 dist-test-slave-7c35 krb5kdc[24654](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016188, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T11:36:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T11:36:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.14.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab.
Entry for principal kudu/127.21.14.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab.
I20260806 11:36:28.247318 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786016169045816-21563-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786016169045816-21563-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786016169045816-21563-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:34081
--webserver_interface=127.21.14.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab
--principal=kudu/127.21.14.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:37331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.14.254:34081
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:36:28.371188 24668 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:36:28.371515 24668 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:36:28.371565 24668 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:36:28.376241 24668 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 11:36:28.376322 24668 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:36:28.376344 24668 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:36:28.376362 24668 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 11:36:28.376379 24668 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 11:36:28.382557 24668 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:37331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786016169045816-21563-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786016169045816-21563-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.14.254:34081
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab
--principal=kudu/127.21.14.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:34081
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.14.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24668
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786016169045816-21563-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:36:28.384742 24668 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:36:28.386073 24668 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:36:28.396294 24674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:28.396413 24668 server_base.cc:1034] running on GCE node
W20260806 11:36:28.396395 24673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:28.396294 24676 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:28.397269 24668 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:36:28.398555 24668 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:36:28.399747 24668 hybrid_clock.cc:648] HybridClock initialized: now 1786016188399710 us; error 53 us; skew 500 ppm
Aug 06 11:36:28 dist-test-slave-7c35 krb5kdc[24654](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016188, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:36:28.408778 24668 init.cc:377] Logged in from keytab as kudu/127.21.14.254@KRBTEST.COM (short username kudu)
I20260806 11:36:28.409787 24668 webserver.cc:460] Webserver started at http://127.21.14.254:33599/ using document root <none> and password file <none>
I20260806 11:36:28.410458 24668 fs_manager.cc:359] Metadata directory not provided
I20260806 11:36:28.410550 24668 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:36:28.410741 24668 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:36:28.413106 24668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786016169045816-21563-0/minicluster-data/master-0/data/instance:
uuid: "9aa5d57d2c3f44b2b6b5ee0fef244051"
format_stamp: "Formatted at 2026-08-06 11:36:28 on dist-test-slave-7c35"
server_key: "f46c58f3520d6f9b65c30eb3342e30a3"
server_key_iv: "46a701e2e05e46f8184414e8d5a3f0df"
server_key_version: "encryptionkey@0"
I20260806 11:36:28.413669 24668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786016169045816-21563-0/minicluster-data/master-0/wal/instance:
uuid: "9aa5d57d2c3f44b2b6b5ee0fef244051"
format_stamp: "Formatted at 2026-08-06 11:36:28 on dist-test-slave-7c35"
server_key: "f46c58f3520d6f9b65c30eb3342e30a3"
server_key_iv: "46a701e2e05e46f8184414e8d5a3f0df"
server_key_version: "encryptionkey@0"
I20260806 11:36:28.417763 24668 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 11:36:28.420485 24685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:28.421866 24668 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 11:36:28.422022 24668 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786016169045816-21563-0/minicluster-data/master-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786016169045816-21563-0/minicluster-data/master-0/wal
uuid: "9aa5d57d2c3f44b2b6b5ee0fef244051"
format_stamp: "Formatted at 2026-08-06 11:36:28 on dist-test-slave-7c35"
server_key: "f46c58f3520d6f9b65c30eb3342e30a3"
server_key_iv: "46a701e2e05e46f8184414e8d5a3f0df"
server_key_version: "encryptionkey@0"
I20260806 11:36:28.422158 24668 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786016169045816-21563-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786016169045816-21563-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786016169045816-21563-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:36:28.435346 24668 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:36:28.439162 24668 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:36:28.439405 24668 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:36:28.447329 24668 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.254:34081
I20260806 11:36:28.447695 24739 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.254:34081 every 8 connection(s)
I20260806 11:36:28.448423 24668 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
I20260806 11:36:28.451565 24740 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:28.456275 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 24668
I20260806 11:36:28.456414 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786016169045816-21563-0/minicluster-data/master-0/wal/instance
I20260806 11:36:28.456693 21563 external_mini_cluster.cc:1428] Setting key de4672d9782745b14fe924991e041a89
I20260806 11:36:28.458362 24740 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9aa5d57d2c3f44b2b6b5ee0fef244051: Bootstrap starting.
I20260806 11:36:28.461198 24740 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9aa5d57d2c3f44b2b6b5ee0fef244051: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:28.462066 24740 log.cc:824] T 00000000000000000000000000000000 P 9aa5d57d2c3f44b2b6b5ee0fef244051: Log is configured to *not* fsync() on all Append() calls
I20260806 11:36:28.464624 24740 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9aa5d57d2c3f44b2b6b5ee0fef244051: No bootstrap required, opened a new log
Aug 06 11:36:28 dist-test-slave-7c35 krb5kdc[24654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016188, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:36:28.470304 24740 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9aa5d57d2c3f44b2b6b5ee0fef244051 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9aa5d57d2c3f44b2b6b5ee0fef244051" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 34081 } }
I20260806 11:36:28.470662 24740 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9aa5d57d2c3f44b2b6b5ee0fef244051 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:28.470736 24740 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9aa5d57d2c3f44b2b6b5ee0fef244051 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aa5d57d2c3f44b2b6b5ee0fef244051, State: Initialized, Role: FOLLOWER
I20260806 11:36:28.471380 24740 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9aa5d57d2c3f44b2b6b5ee0fef244051 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9aa5d57d2c3f44b2b6b5ee0fef244051" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 34081 } }
I20260806 11:36:28.471544 24740 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9aa5d57d2c3f44b2b6b5ee0fef244051 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 11:36:28.471630 24740 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9aa5d57d2c3f44b2b6b5ee0fef244051 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 11:36:28.471771 24740 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9aa5d57d2c3f44b2b6b5ee0fef244051 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:36:28.472855 24740 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9aa5d57d2c3f44b2b6b5ee0fef244051 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9aa5d57d2c3f44b2b6b5ee0fef244051" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 34081 } }
I20260806 11:36:28.473263 24740 leader_election.cc:302] T 00000000000000000000000000000000 P 9aa5d57d2c3f44b2b6b5ee0fef244051 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9aa5d57d2c3f44b2b6b5ee0fef244051; no voters:
I20260806 11:36:28.473651 24740 leader_election.cc:288] T 00000000000000000000000000000000 P 9aa5d57d2c3f44b2b6b5ee0fef244051 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 11:36:28.473752 24746 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9aa5d57d2c3f44b2b6b5ee0fef244051 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:36:28.473953 24746 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9aa5d57d2c3f44b2b6b5ee0fef244051 [term 1 LEADER]: Becoming Leader. State: Replica: 9aa5d57d2c3f44b2b6b5ee0fef244051, State: Running, Role: LEADER
I20260806 11:36:28.474363 24746 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9aa5d57d2c3f44b2b6b5ee0fef244051 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9aa5d57d2c3f44b2b6b5ee0fef244051" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 34081 } }
I20260806 11:36:28.474606 24744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:28.458255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60728 (local address 127.21.14.254:34081)
0806 11:36:28.458765 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:28.458783 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:28.458823 (+ 40us) server_negotiation.cc:407] Connection header received
0806 11:36:28.459585 (+ 762us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:28.459604 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:28.459957 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:28.460264 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:28.461368 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:28.462520 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:28.463593 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:28.464321 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:28.464500 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:28.467533 (+ 3033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:28.467541 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:28.467564 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:28.467607 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:28.470307 (+ 2700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:28.471088 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:28.471096 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:28.471102 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:28.471271 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:28.471752 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:28.471756 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:28.471757 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:28.472333 (+ 576us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:28.472598 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:28.472948 (+ 350us) server_negotiation.cc:299] Negotiation successful
0806 11:36:28.473337 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":183,"threads_started":1}
I20260806 11:36:28.475315 24740 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9aa5d57d2c3f44b2b6b5ee0fef244051 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 11:36:28.476277 24748 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9aa5d57d2c3f44b2b6b5ee0fef244051 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9aa5d57d2c3f44b2b6b5ee0fef244051. Latest consensus state: current_term: 1 leader_uuid: "9aa5d57d2c3f44b2b6b5ee0fef244051" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9aa5d57d2c3f44b2b6b5ee0fef244051" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 34081 } } }
I20260806 11:36:28.476426 24748 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9aa5d57d2c3f44b2b6b5ee0fef244051 [sys.catalog]: This master's current role is: LEADER
I20260806 11:36:28.476774 24747 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9aa5d57d2c3f44b2b6b5ee0fef244051 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9aa5d57d2c3f44b2b6b5ee0fef244051" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9aa5d57d2c3f44b2b6b5ee0fef244051" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 34081 } } }
I20260806 11:36:28.476861 24747 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9aa5d57d2c3f44b2b6b5ee0fef244051 [sys.catalog]: This master's current role is: LEADER
I20260806 11:36:28.477312 24755 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 11:36:28.478229 21563 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 11:36:28.480854 24755 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 11:36:28.488101 24755 catalog_manager.cc:1347] Generated new cluster ID: 0a26e88ec3634c56bd07451f699c5cb0
I20260806 11:36:28.488296 24755 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 11:36:28.509543 24755 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 11:36:28.509912 24744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:28.503440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60740 (local address 127.21.14.254:34081)
0806 11:36:28.503594 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:28.503598 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:28.503875 (+ 277us) server_negotiation.cc:407] Connection header received
0806 11:36:28.504065 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:28.504068 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:28.504118 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:28.504208 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:28.504776 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:28.505324 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:28.506240 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:28.506555 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:28.506664 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:28.508584 (+ 1920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:28.508590 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:28.508595 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 11:36:28.509316 (+ 721us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:28.509363 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 11:36:28.509387 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:28.509506 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:28.509689 (+ 183us) server_negotiation.cc:299] Negotiation successful
0806 11:36:28.509772 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 11:36:28.512305 24755 catalog_manager.cc:1504] Loading token signing keys...
I20260806 11:36:28.527896 24755 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9aa5d57d2c3f44b2b6b5ee0fef244051: Generated new TSK 0
I20260806 11:36:28.528898 24755 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 11:36:28.539690 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 24668
2026-08-06T11:36:28Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1552 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 11:36:28 dist-test-slave-7c35 krb5kdc[24775](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 11:36:28 dist-test-slave-7c35 krb5kdc[24775](info): set up 2 sockets
Aug 06 11:36:28 dist-test-slave-7c35 krb5kdc[24775](info): commencing operation
krb5kdc: starting...
W20260806 11:36:29.643779 21563 mini_kdc.cc:121] Time spent starting KDC: real 1.081s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Aug 06 11:36:29 dist-test-slave-7c35 krb5kdc[24775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016189, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T11:36:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T11:36:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.14.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab.
Entry for principal kudu/127.21.14.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab.
I20260806 11:36:29.792084 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:35853
--webserver_interface=127.21.14.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab
--principal=kudu/127.21.14.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:46003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.14.254:35853
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:36:29.912519 24789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:36:29.912827 24789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:36:29.912870 24789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:36:29.917258 24789 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 11:36:29.917342 24789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:36:29.917362 24789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:36:29.917379 24789 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 11:36:29.917395 24789 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 11:36:29.923187 24789 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:46003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.14.254:35853
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab
--principal=kudu/127.21.14.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:35853
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.14.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24789
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:36:29.924365 24789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:36:29.925469 24789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 11:36:29.935532 24789 server_base.cc:1034] running on GCE node
W20260806 11:36:29.935491 24795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:29.935499 24797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:29.935496 24794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:29.936610 24789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:36:29.937841 24789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:36:29.939018 24789 hybrid_clock.cc:648] HybridClock initialized: now 1786016189938999 us; error 43 us; skew 500 ppm
Aug 06 11:36:29 dist-test-slave-7c35 krb5kdc[24775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016189, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:36:29.942152 24789 init.cc:377] Logged in from keytab as kudu/127.21.14.254@KRBTEST.COM (short username kudu)
I20260806 11:36:29.943336 24789 webserver.cc:460] Webserver started at http://127.21.14.254:35177/ using document root <none> and password file <none>
I20260806 11:36:29.943981 24789 fs_manager.cc:359] Metadata directory not provided
I20260806 11:36:29.944083 24789 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:36:29.944337 24789 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:36:29.946641 24789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/minicluster-data/master-0/data/instance:
uuid: "6b4f8df27eec4ef7967fe1f6da6321bc"
format_stamp: "Formatted at 2026-08-06 11:36:29 on dist-test-slave-7c35"
server_key: "369e0acaf03cf5d25701b9c983f42be7"
server_key_iv: "e06bb288fbed0e2af247b84575989a28"
server_key_version: "encryptionkey@0"
I20260806 11:36:29.947319 24789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/minicluster-data/master-0/wal/instance:
uuid: "6b4f8df27eec4ef7967fe1f6da6321bc"
format_stamp: "Formatted at 2026-08-06 11:36:29 on dist-test-slave-7c35"
server_key: "369e0acaf03cf5d25701b9c983f42be7"
server_key_iv: "e06bb288fbed0e2af247b84575989a28"
server_key_version: "encryptionkey@0"
I20260806 11:36:29.951555 24789 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 11:36:29.954296 24804 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:29.955819 24789 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 11:36:29.956000 24789 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/minicluster-data/master-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/minicluster-data/master-0/wal
uuid: "6b4f8df27eec4ef7967fe1f6da6321bc"
format_stamp: "Formatted at 2026-08-06 11:36:29 on dist-test-slave-7c35"
server_key: "369e0acaf03cf5d25701b9c983f42be7"
server_key_iv: "e06bb288fbed0e2af247b84575989a28"
server_key_version: "encryptionkey@0"
I20260806 11:36:29.956146 24789 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:36:29.978525 24789 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:36:29.982347 24789 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:36:29.982606 24789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:36:29.990356 24789 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.254:35853
I20260806 11:36:29.990674 24861 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.254:35853 every 8 connection(s)
I20260806 11:36:29.991508 24789 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
I20260806 11:36:29.994665 24862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:30.000589 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 24789
I20260806 11:36:30.000736 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/minicluster-data/master-0/wal/instance
I20260806 11:36:30.001145 21563 external_mini_cluster.cc:1428] Setting key 1cb420e0da16dff87d2b93e3a9de01cd
I20260806 11:36:30.001798 24862 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b4f8df27eec4ef7967fe1f6da6321bc: Bootstrap starting.
I20260806 11:36:30.004940 24862 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6b4f8df27eec4ef7967fe1f6da6321bc: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:30.006026 24862 log.cc:824] T 00000000000000000000000000000000 P 6b4f8df27eec4ef7967fe1f6da6321bc: Log is configured to *not* fsync() on all Append() calls
I20260806 11:36:30.008581 24862 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b4f8df27eec4ef7967fe1f6da6321bc: No bootstrap required, opened a new log
Aug 06 11:36:30 dist-test-slave-7c35 krb5kdc[24775](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016189, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:36:30.013221 24862 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6b4f8df27eec4ef7967fe1f6da6321bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b4f8df27eec4ef7967fe1f6da6321bc" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 35853 } }
I20260806 11:36:30.013509 24862 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6b4f8df27eec4ef7967fe1f6da6321bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:30.013602 24862 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6b4f8df27eec4ef7967fe1f6da6321bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b4f8df27eec4ef7967fe1f6da6321bc, State: Initialized, Role: FOLLOWER
I20260806 11:36:30.014081 24862 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b4f8df27eec4ef7967fe1f6da6321bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b4f8df27eec4ef7967fe1f6da6321bc" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 35853 } }
I20260806 11:36:30.014240 24862 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6b4f8df27eec4ef7967fe1f6da6321bc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 11:36:30.014349 24862 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6b4f8df27eec4ef7967fe1f6da6321bc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 11:36:30.014464 24862 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b4f8df27eec4ef7967fe1f6da6321bc [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:36:30.015748 24862 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6b4f8df27eec4ef7967fe1f6da6321bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b4f8df27eec4ef7967fe1f6da6321bc" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 35853 } }
I20260806 11:36:30.016162 24862 leader_election.cc:302] T 00000000000000000000000000000000 P 6b4f8df27eec4ef7967fe1f6da6321bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6b4f8df27eec4ef7967fe1f6da6321bc; no voters:
I20260806 11:36:30.016551 24862 leader_election.cc:288] T 00000000000000000000000000000000 P 6b4f8df27eec4ef7967fe1f6da6321bc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 11:36:30.017005 24868 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6b4f8df27eec4ef7967fe1f6da6321bc [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:36:30.017251 24868 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6b4f8df27eec4ef7967fe1f6da6321bc [term 1 LEADER]: Becoming Leader. State: Replica: 6b4f8df27eec4ef7967fe1f6da6321bc, State: Running, Role: LEADER
I20260806 11:36:30.017652 24868 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6b4f8df27eec4ef7967fe1f6da6321bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b4f8df27eec4ef7967fe1f6da6321bc" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 35853 } }
I20260806 11:36:30.017904 24865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:30.002756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53134 (local address 127.21.14.254:35853)
0806 11:36:30.003349 (+ 593us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:30.003368 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:30.003407 (+ 39us) server_negotiation.cc:407] Connection header received
0806 11:36:30.004151 (+ 744us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:30.004170 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:30.004553 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:30.004857 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:30.005885 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:30.007103 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:30.008319 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:30.008878 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:30.009004 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:30.012054 (+ 3050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:30.012061 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:30.012078 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:30.012113 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:30.014178 (+ 2065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:30.014766 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:30.014771 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:30.014776 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:30.014965 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:30.015296 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:30.015299 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:30.015300 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:30.015780 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:30.016032 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:30.016362 (+ 330us) server_negotiation.cc:299] Negotiation successful
0806 11:36:30.016758 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":425,"thread_start_us":188,"threads_started":1}
I20260806 11:36:30.017987 24862 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6b4f8df27eec4ef7967fe1f6da6321bc [sys.catalog]: configured and running, proceeding with master startup.
I20260806 11:36:30.020591 24869 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b4f8df27eec4ef7967fe1f6da6321bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6b4f8df27eec4ef7967fe1f6da6321bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b4f8df27eec4ef7967fe1f6da6321bc" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 35853 } } }
I20260806 11:36:30.020766 24869 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b4f8df27eec4ef7967fe1f6da6321bc [sys.catalog]: This master's current role is: LEADER
I20260806 11:36:30.020810 24867 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b4f8df27eec4ef7967fe1f6da6321bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6b4f8df27eec4ef7967fe1f6da6321bc. Latest consensus state: current_term: 1 leader_uuid: "6b4f8df27eec4ef7967fe1f6da6321bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b4f8df27eec4ef7967fe1f6da6321bc" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 35853 } } }
I20260806 11:36:30.020911 24867 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b4f8df27eec4ef7967fe1f6da6321bc [sys.catalog]: This master's current role is: LEADER
I20260806 11:36:30.021853 24877 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 11:36:30.025218 24877 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 11:36:30.032089 24877 catalog_manager.cc:1347] Generated new cluster ID: 929ad14fcc8f46caa0873478378f963d
I20260806 11:36:30.032208 24877 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 11:36:30.054293 24877 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 11:36:30.055605 24877 catalog_manager.cc:1504] Loading token signing keys...
I20260806 11:36:30.070585 24877 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6b4f8df27eec4ef7967fe1f6da6321bc: Generated new TSK 0
I20260806 11:36:30.071504 24877 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.14.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab.
Entry for principal kudu/127.21.14.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab.
I20260806 11:36:30.119858 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.193:0
--local_ip_for_outbound_sockets=127.21.14.193
--webserver_interface=127.21.14.193
--webserver_port=0
--tserver_master_addrs=127.21.14.254:35853
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab
--principal=kudu/127.21.14.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:46003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:36:30.260871 24889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:36:30.261281 24889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:36:30.261359 24889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:36:30.265766 24889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:36:30.265848 24889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:36:30.265952 24889 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.193
I20260806 11:36:30.271927 24889 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:46003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab
--principal=kudu/127.21.14.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.14.193
--webserver_port=0
--tserver_master_addrs=127.21.14.254:35853
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24889
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.193
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:36:30.273211 24889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:36:30.274338 24889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 11:36:30.286367 24889 server_base.cc:1034] running on GCE node
W20260806 11:36:30.286296 24895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:30.286300 24894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:30.286324 24898 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:30.287156 24889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:36:30.287949 24889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:36:30.289171 24889 hybrid_clock.cc:648] HybridClock initialized: now 1786016190289109 us; error 55 us; skew 500 ppm
Aug 06 11:36:30 dist-test-slave-7c35 krb5kdc[24775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016190, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:36:30.292363 24889 init.cc:377] Logged in from keytab as kudu/127.21.14.193@KRBTEST.COM (short username kudu)
I20260806 11:36:30.293412 24889 webserver.cc:460] Webserver started at http://127.21.14.193:44923/ using document root <none> and password file <none>
I20260806 11:36:30.294065 24889 fs_manager.cc:359] Metadata directory not provided
I20260806 11:36:30.294171 24889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:36:30.294430 24889 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:36:30.296792 24889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-0/data/instance:
uuid: "d3a1ee72e3ad46d883a316b7603d8bf0"
format_stamp: "Formatted at 2026-08-06 11:36:30 on dist-test-slave-7c35"
server_key: "aab3f6db8ea1c3630c867243362a4c6e"
server_key_iv: "56d645d5e777b05dce2d0edf19e02cef"
server_key_version: "encryptionkey@0"
I20260806 11:36:30.297443 24889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-0/wal/instance:
uuid: "d3a1ee72e3ad46d883a316b7603d8bf0"
format_stamp: "Formatted at 2026-08-06 11:36:30 on dist-test-slave-7c35"
server_key: "aab3f6db8ea1c3630c867243362a4c6e"
server_key_iv: "56d645d5e777b05dce2d0edf19e02cef"
server_key_version: "encryptionkey@0"
I20260806 11:36:30.302083 24889 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260806 11:36:30.305338 24906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:30.307034 24889 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 11:36:30.307215 24889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-0/wal
uuid: "d3a1ee72e3ad46d883a316b7603d8bf0"
format_stamp: "Formatted at 2026-08-06 11:36:30 on dist-test-slave-7c35"
server_key: "aab3f6db8ea1c3630c867243362a4c6e"
server_key_iv: "56d645d5e777b05dce2d0edf19e02cef"
server_key_version: "encryptionkey@0"
I20260806 11:36:30.307366 24889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:36:30.320639 24889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:36:30.324509 24889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:36:30.324746 24889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:36:30.325431 24889 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:36:30.326377 24889 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:36:30.326452 24889 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:30.326514 24889 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:36:30.326570 24889 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:30.337815 24889 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.193:44399
I20260806 11:36:30.338012 25029 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.193:44399 every 8 connection(s)
I20260806 11:36:30.339231 24889 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
I20260806 11:36:30.339627 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 24889
I20260806 11:36:30.339762 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-0/wal/instance
I20260806 11:36:30.340122 21563 external_mini_cluster.cc:1428] Setting key 8099dcf1a48be94926ac58691c006644
WARNING: no policy specified for kudu/127.21.14.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.194@KRBTEST.COM" created.
Aug 06 11:36:30 dist-test-slave-7c35 krb5kdc[24775](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016190, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.193@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:36:30.363409 24865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:30.341584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:53359 (local address 127.21.14.254:35853)
0806 11:36:30.341702 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:30.341706 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:30.343182 (+ 1476us) server_negotiation.cc:407] Connection header received
0806 11:36:30.344870 (+ 1688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:30.344874 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:30.344950 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:30.345079 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:30.346943 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:30.347516 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:30.349162 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:30.349433 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:30.349568 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:30.357189 (+ 7621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:30.357192 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:30.357195 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:30.357232 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:30.359597 (+ 2365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:30.360673 (+ 1076us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:30.360677 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:30.360679 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:30.360920 (+ 241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:30.361465 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:30.361469 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:30.361470 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:30.361939 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:30.362091 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:30.363041 (+ 950us) server_negotiation.cc:299] Negotiation successful
0806 11:36:30.363194 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 11:36:30.364780 25032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:30.342159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:35853 (local address 127.21.14.193:53359)
0806 11:36:30.342822 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:30.343047 (+ 225us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:30.344570 (+ 1523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:30.345321 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:30.345337 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:30.345782 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:30.346657 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:30.346677 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:30.347686 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:30.347691 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:30.348963 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:30.348979 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:30.349640 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:30.349645 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:30.349856 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:30.350672 (+ 816us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:30.350701 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:30.356859 (+ 6158us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:30.359962 (+ 3103us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:30.359979 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:30.360009 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:30.360524 (+ 515us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:30.361056 (+ 532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:30.361062 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:30.361067 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:30.361326 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:30.362095 (+ 769us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:30.362105 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:30.362578 (+ 473us) client_negotiation.cc:769] Sending connection context
0806 11:36:30.362876 (+ 298us) client_negotiation.cc:240] Negotiation successful
0806 11:36:30.363362 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":180,"threads_started":1}
I20260806 11:36:30.366504 25030 heartbeater.cc:347] Connected to a master server at 127.21.14.254:35853
I20260806 11:36:30.366816 25030 heartbeater.cc:464] Registering TS with master...
I20260806 11:36:30.367643 25030 heartbeater.cc:511] Master 127.21.14.254:35853 requested a full tablet report, sending...
I20260806 11:36:30.369701 24822 ts_manager.cc:194] Registered new tserver with Master: d3a1ee72e3ad46d883a316b7603d8bf0 (127.21.14.193:44399)
I20260806 11:36:30.371356 24822 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.14.193@KRBTEST.COM'} at 127.21.14.193:53359
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab.
Entry for principal kudu/127.21.14.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab.
I20260806 11:36:30.379689 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.194:0
--local_ip_for_outbound_sockets=127.21.14.194
--webserver_interface=127.21.14.194
--webserver_port=0
--tserver_master_addrs=127.21.14.254:35853
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab
--principal=kudu/127.21.14.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:46003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:36:30.500483 25035 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:36:30.500841 25035 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:36:30.500952 25035 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:36:30.505295 25035 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:36:30.505403 25035 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:36:30.505504 25035 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.194
I20260806 11:36:30.512378 25035 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:46003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab
--principal=kudu/127.21.14.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.14.194
--webserver_port=0
--tserver_master_addrs=127.21.14.254:35853
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25035
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.194
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:36:30.513710 25035 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:36:30.514941 25035 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:36:30.527063 25041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:30.527067 25040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:30.527093 25043 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:30.527350 25035 server_base.cc:1034] running on GCE node
I20260806 11:36:30.527932 25035 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:36:30.528661 25035 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:36:30.529870 25035 hybrid_clock.cc:648] HybridClock initialized: now 1786016190529838 us; error 44 us; skew 500 ppm
Aug 06 11:36:30 dist-test-slave-7c35 krb5kdc[24775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016190, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:36:30.533059 25035 init.cc:377] Logged in from keytab as kudu/127.21.14.194@KRBTEST.COM (short username kudu)
I20260806 11:36:30.534103 25035 webserver.cc:460] Webserver started at http://127.21.14.194:46023/ using document root <none> and password file <none>
I20260806 11:36:30.534730 25035 fs_manager.cc:359] Metadata directory not provided
I20260806 11:36:30.534826 25035 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:36:30.535096 25035 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:36:30.537293 25035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-1/data/instance:
uuid: "ef79675d22a14016be0b48333263bd8f"
format_stamp: "Formatted at 2026-08-06 11:36:30 on dist-test-slave-7c35"
server_key: "efd354e86fdbd9c4b2916933d96689ec"
server_key_iv: "fd391181e211f06a573599e087260cc3"
server_key_version: "encryptionkey@0"
I20260806 11:36:30.537905 25035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-1/wal/instance:
uuid: "ef79675d22a14016be0b48333263bd8f"
format_stamp: "Formatted at 2026-08-06 11:36:30 on dist-test-slave-7c35"
server_key: "efd354e86fdbd9c4b2916933d96689ec"
server_key_iv: "fd391181e211f06a573599e087260cc3"
server_key_version: "encryptionkey@0"
I20260806 11:36:30.541973 25035 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260806 11:36:30.544838 25050 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:30.546196 25035 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 11:36:30.546365 25035 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-1/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-1/wal
uuid: "ef79675d22a14016be0b48333263bd8f"
format_stamp: "Formatted at 2026-08-06 11:36:30 on dist-test-slave-7c35"
server_key: "efd354e86fdbd9c4b2916933d96689ec"
server_key_iv: "fd391181e211f06a573599e087260cc3"
server_key_version: "encryptionkey@0"
I20260806 11:36:30.546497 25035 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:36:30.565654 25035 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:36:30.569344 25035 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:36:30.569569 25035 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:36:30.570303 25035 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:36:30.571269 25035 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:36:30.571341 25035 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:30.571415 25035 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:36:30.571465 25035 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:30.582154 25035 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.194:33689
I20260806 11:36:30.582186 25171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.194:33689 every 8 connection(s)
I20260806 11:36:30.583251 25035 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
I20260806 11:36:30.588449 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 25035
I20260806 11:36:30.588589 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-1/wal/instance
I20260806 11:36:30.588860 21563 external_mini_cluster.cc:1428] Setting key c5f97ec245f1f3ee98bb4319f34ca3c6
Aug 06 11:36:30 dist-test-slave-7c35 krb5kdc[24775](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016190, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.194@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:36:30.600383 24865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:30.585318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:40843 (local address 127.21.14.254:35853)
0806 11:36:30.585504 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:30.585510 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:30.586412 (+ 902us) server_negotiation.cc:407] Connection header received
0806 11:36:30.587913 (+ 1501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:30.587916 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:30.587968 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:30.588045 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:30.589705 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:30.590367 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:30.591782 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:30.592030 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:30.592154 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:30.595367 (+ 3213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:30.595370 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:30.595372 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:30.595407 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:30.597581 (+ 2174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:30.598287 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:30.598294 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:30.598297 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:30.598413 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:30.598814 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:30.598818 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:30.598819 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:30.599149 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:30.599289 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:30.599948 (+ 659us) server_negotiation.cc:299] Negotiation successful
0806 11:36:30.600122 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
WARNING: no policy specified for kudu/127.21.14.195@KRBTEST.COM; defaulting to no policy
I20260806 11:36:30.601230 25174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:30.585753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:35853 (local address 127.21.14.194:40843)
0806 11:36:30.586246 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:30.586292 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:30.587684 (+ 1392us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:30.588244 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:30.588252 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:30.588764 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:30.589445 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:30.589472 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:30.590524 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:30.590527 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:30.591630 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:30.591642 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:30.592208 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:30.592213 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:30.592416 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:30.593231 (+ 815us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:30.593259 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:30.595143 (+ 1884us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:30.597796 (+ 2653us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:30.597803 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:30.597820 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:30.598141 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:30.598519 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:30.598522 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:30.598523 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:30.598694 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:30.599311 (+ 617us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:30.599317 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:30.599639 (+ 322us) client_negotiation.cc:769] Sending connection context
0806 11:36:30.599821 (+ 182us) client_negotiation.cc:240] Negotiation successful
0806 11:36:30.600139 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":133,"threads_started":1}
I20260806 11:36:30.602741 25172 heartbeater.cc:347] Connected to a master server at 127.21.14.254:35853
I20260806 11:36:30.603101 25172 heartbeater.cc:464] Registering TS with master...
I20260806 11:36:30.603653 25172 heartbeater.cc:511] Master 127.21.14.254:35853 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.195@KRBTEST.COM" created.
I20260806 11:36:30.605096 24822 ts_manager.cc:194] Registered new tserver with Master: ef79675d22a14016be0b48333263bd8f (127.21.14.194:33689)
I20260806 11:36:30.605788 24822 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.14.194@KRBTEST.COM'} at 127.21.14.194:40843
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab.
Entry for principal kudu/127.21.14.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab.
I20260806 11:36:30.628132 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.195:0
--local_ip_for_outbound_sockets=127.21.14.195
--webserver_interface=127.21.14.195
--webserver_port=0
--tserver_master_addrs=127.21.14.254:35853
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab
--principal=kudu/127.21.14.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:46003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:36:30.748951 25177 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:36:30.749306 25177 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:36:30.749390 25177 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:36:30.753858 25177 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:36:30.753973 25177 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:36:30.754076 25177 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.195
I20260806 11:36:30.759867 25177 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:46003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab
--principal=kudu/127.21.14.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.14.195
--webserver_port=0
--tserver_master_addrs=127.21.14.254:35853
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25177
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.195
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:36:30.761211 25177 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:36:30.762353 25177 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:36:30.772954 25183 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:30.773051 25177 server_base.cc:1034] running on GCE node
W20260806 11:36:30.773006 25186 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:30.772969 25184 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:30.773720 25177 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:36:30.774391 25177 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:36:30.775588 25177 hybrid_clock.cc:648] HybridClock initialized: now 1786016190775547 us; error 53 us; skew 500 ppm
Aug 06 11:36:30 dist-test-slave-7c35 krb5kdc[24775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016190, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:36:30.778940 25177 init.cc:377] Logged in from keytab as kudu/127.21.14.195@KRBTEST.COM (short username kudu)
I20260806 11:36:30.780035 25177 webserver.cc:460] Webserver started at http://127.21.14.195:38497/ using document root <none> and password file <none>
I20260806 11:36:30.780660 25177 fs_manager.cc:359] Metadata directory not provided
I20260806 11:36:30.780758 25177 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:36:30.781013 25177 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:36:30.783799 25177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-2/data/instance:
uuid: "1f6feb36d04a4435b3d85fce6af9f6bf"
format_stamp: "Formatted at 2026-08-06 11:36:30 on dist-test-slave-7c35"
server_key: "4c480a488b26a0341f8b3640ac1581c5"
server_key_iv: "e7a0fe6d6d6bdaeae6c02e538ac92b46"
server_key_version: "encryptionkey@0"
I20260806 11:36:30.784540 25177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-2/wal/instance:
uuid: "1f6feb36d04a4435b3d85fce6af9f6bf"
format_stamp: "Formatted at 2026-08-06 11:36:30 on dist-test-slave-7c35"
server_key: "4c480a488b26a0341f8b3640ac1581c5"
server_key_iv: "e7a0fe6d6d6bdaeae6c02e538ac92b46"
server_key_version: "encryptionkey@0"
I20260806 11:36:30.789428 25177 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 11:36:30.792446 25193 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:30.793991 25177 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 11:36:30.794163 25177 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-2/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-2/wal
uuid: "1f6feb36d04a4435b3d85fce6af9f6bf"
format_stamp: "Formatted at 2026-08-06 11:36:30 on dist-test-slave-7c35"
server_key: "4c480a488b26a0341f8b3640ac1581c5"
server_key_iv: "e7a0fe6d6d6bdaeae6c02e538ac92b46"
server_key_version: "encryptionkey@0"
I20260806 11:36:30.794302 25177 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:36:30.808346 25177 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:36:30.812646 25177 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:36:30.812824 25177 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:36:30.813395 25177 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:36:30.814253 25177 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:36:30.814304 25177 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:30.814347 25177 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:36:30.814370 25177 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:30.823855 25177 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.195:44491
I20260806 11:36:30.823936 25314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.195:44491 every 8 connection(s)
I20260806 11:36:30.824880 25177 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
I20260806 11:36:30.826473 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 25177
I20260806 11:36:30.826624 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786016169045816-21563-0/minicluster-data/ts-2/wal/instance
I20260806 11:36:30.826982 21563 external_mini_cluster.cc:1428] Setting key 66622062a10c8a1e35a11c6a863fabef
Aug 06 11:36:30 dist-test-slave-7c35 krb5kdc[24775](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016190, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.195@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:36:30.840881 24865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:30.827369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:50799 (local address 127.21.14.254:35853)
0806 11:36:30.827549 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:30.827553 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:30.828623 (+ 1070us) server_negotiation.cc:407] Connection header received
0806 11:36:30.830349 (+ 1726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:30.830352 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:30.830401 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:30.830470 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:30.831862 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:30.832436 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:30.833780 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:30.834011 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:30.834194 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:30.837134 (+ 2940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:30.837138 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:30.837140 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:30.837179 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:30.838527 (+ 1348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:30.839224 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:30.839227 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:30.839229 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:30.839315 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:30.839654 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:30.839656 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:30.839657 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:30.839871 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:30.839946 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:30.840575 (+ 629us) server_negotiation.cc:299] Negotiation successful
0806 11:36:30.840715 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 11:36:30.841976 25317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:30.827768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:35853 (local address 127.21.14.195:50799)
0806 11:36:30.828450 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:30.828517 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:30.830120 (+ 1603us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:30.830621 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:30.830629 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:30.831132 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:30.831674 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:30.831689 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:30.832601 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:30.832604 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:30.833628 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:30.833639 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:30.834244 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:30.834248 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:30.834422 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:30.835183 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:30.835203 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:30.836921 (+ 1718us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:30.838688 (+ 1767us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:30.838694 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:30.838711 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:30.839106 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:30.839408 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:30.839410 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:30.839411 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:30.839563 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:30.839973 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:30.839981 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:30.840303 (+ 322us) client_negotiation.cc:769] Sending connection context
0806 11:36:30.840506 (+ 203us) client_negotiation.cc:240] Negotiation successful
0806 11:36:30.840906 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":165,"threads_started":1}
I20260806 11:36:30.843389 25315 heartbeater.cc:347] Connected to a master server at 127.21.14.254:35853
I20260806 11:36:30.843665 25315 heartbeater.cc:464] Registering TS with master...
I20260806 11:36:30.844210 25315 heartbeater.cc:511] Master 127.21.14.254:35853 requested a full tablet report, sending...
I20260806 11:36:30.845408 24822 ts_manager.cc:194] Registered new tserver with Master: 1f6feb36d04a4435b3d85fce6af9f6bf (127.21.14.195:44491)
I20260806 11:36:30.846131 24822 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.14.195@KRBTEST.COM'} at 127.21.14.195:50799
I20260806 11:36:30.854131 21563 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 11:36:30 dist-test-slave-7c35 krb5kdc[24775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016190, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 06 11:36:30 dist-test-slave-7c35 krb5kdc[24775](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016190, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:36:30.883473 24865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:30.873496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53146 (local address 127.21.14.254:35853)
0806 11:36:30.873710 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:30.873715 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:30.873888 (+ 173us) server_negotiation.cc:407] Connection header received
0806 11:36:30.874078 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:30.874082 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:30.874130 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:30.874224 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:30.875066 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:30.875633 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:30.876551 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:30.876781 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:30.876901 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:30.879672 (+ 2771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:30.879676 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:30.879678 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:30.879708 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:30.881263 (+ 1555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:30.881912 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:30.881918 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:30.881920 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:30.882026 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:30.882356 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:30.882360 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:30.882362 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:30.882643 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:30.882834 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:30.883124 (+ 290us) server_negotiation.cc:299] Negotiation successful
0806 11:36:30.883267 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
W20260806 11:36:30.883951 24822 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:53146
I20260806 11:36:30.885004 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 24889
I20260806 11:36:30.893090 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 25035
I20260806 11:36:30.900338 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 25177
I20260806 11:36:30.908126 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 24789
2026-08-06T11:36:30Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2370 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 11:36:30 dist-test-slave-7c35 krb5kdc[25329](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 11:36:30 dist-test-slave-7c35 krb5kdc[25329](info): set up 2 sockets
Aug 06 11:36:30 dist-test-slave-7c35 krb5kdc[25329](info): commencing operation
krb5kdc: starting...
W20260806 11:36:32.011010 21563 mini_kdc.cc:121] Time spent starting KDC: real 1.078s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Aug 06 11:36:32 dist-test-slave-7c35 krb5kdc[25329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016192, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T11:36:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T11:36:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.14.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab.
Entry for principal kudu/127.21.14.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab.
I20260806 11:36:32.175971 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:43645
--webserver_interface=127.21.14.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab
--principal=kudu/127.21.14.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:34395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.14.254:43645
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:36:32.311580 25342 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:36:32.311949 25342 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:36:32.312037 25342 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:36:32.316505 25342 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 11:36:32.316627 25342 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:36:32.316671 25342 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:36:32.316715 25342 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 11:36:32.316756 25342 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 11:36:32.322819 25342 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:34395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.14.254:43645
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab
--principal=kudu/127.21.14.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:43645
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.14.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25342
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:36:32.324213 25342 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:36:32.325408 25342 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:36:32.335253 25347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:32.335280 25350 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:32.335457 25348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:32.335606 25342 server_base.cc:1034] running on GCE node
I20260806 11:36:32.336526 25342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:36:32.337924 25342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:36:32.339138 25342 hybrid_clock.cc:648] HybridClock initialized: now 1786016192339115 us; error 41 us; skew 500 ppm
Aug 06 11:36:32 dist-test-slave-7c35 krb5kdc[25329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016192, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:36:32.342291 25342 init.cc:377] Logged in from keytab as kudu/127.21.14.254@KRBTEST.COM (short username kudu)
I20260806 11:36:32.343406 25342 webserver.cc:460] Webserver started at http://127.21.14.254:43035/ using document root <none> and password file <none>
I20260806 11:36:32.343994 25342 fs_manager.cc:359] Metadata directory not provided
I20260806 11:36:32.344061 25342 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:36:32.344256 25342 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:36:32.346545 25342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/minicluster-data/master-0/data/instance:
uuid: "4421473eff604e4f93fa93547e61b2c0"
format_stamp: "Formatted at 2026-08-06 11:36:32 on dist-test-slave-7c35"
server_key: "e858b2f965556e84d82954cef5b426fd"
server_key_iv: "4baf291f9d3a9991828a3aa8f9e86f22"
server_key_version: "encryptionkey@0"
I20260806 11:36:32.347234 25342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/minicluster-data/master-0/wal/instance:
uuid: "4421473eff604e4f93fa93547e61b2c0"
format_stamp: "Formatted at 2026-08-06 11:36:32 on dist-test-slave-7c35"
server_key: "e858b2f965556e84d82954cef5b426fd"
server_key_iv: "4baf291f9d3a9991828a3aa8f9e86f22"
server_key_version: "encryptionkey@0"
I20260806 11:36:32.351624 25342 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 11:36:32.354385 25357 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:32.355926 25342 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 11:36:32.356106 25342 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/minicluster-data/master-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/minicluster-data/master-0/wal
uuid: "4421473eff604e4f93fa93547e61b2c0"
format_stamp: "Formatted at 2026-08-06 11:36:32 on dist-test-slave-7c35"
server_key: "e858b2f965556e84d82954cef5b426fd"
server_key_iv: "4baf291f9d3a9991828a3aa8f9e86f22"
server_key_version: "encryptionkey@0"
I20260806 11:36:32.356238 25342 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:36:32.369082 25342 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:36:32.373476 25342 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:36:32.373754 25342 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:36:32.382949 25342 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.254:43645
I20260806 11:36:32.382967 25413 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.254:43645 every 8 connection(s)
I20260806 11:36:32.384130 25342 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
I20260806 11:36:32.386334 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 25342
I20260806 11:36:32.386448 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/minicluster-data/master-0/wal/instance
I20260806 11:36:32.386763 21563 external_mini_cluster.cc:1428] Setting key c27298d34f7f44aef2037ee4df9e0cd7
I20260806 11:36:32.387787 25414 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:32.395643 25414 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4421473eff604e4f93fa93547e61b2c0: Bootstrap starting.
Aug 06 11:36:32 dist-test-slave-7c35 krb5kdc[25329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016192, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:36:32.398530 25414 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4421473eff604e4f93fa93547e61b2c0: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:32.399657 25414 log.cc:824] T 00000000000000000000000000000000 P 4421473eff604e4f93fa93547e61b2c0: Log is configured to *not* fsync() on all Append() calls
I20260806 11:36:32.402478 25414 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4421473eff604e4f93fa93547e61b2c0: No bootstrap required, opened a new log
I20260806 11:36:32.404304 25417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:32.388385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46078 (local address 127.21.14.254:43645)
0806 11:36:32.388889 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:32.388911 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:32.388960 (+ 49us) server_negotiation.cc:407] Connection header received
0806 11:36:32.389892 (+ 932us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:32.389918 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:32.390417 (+ 499us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:32.390871 (+ 454us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:32.391741 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:32.393312 (+ 1571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:32.394334 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:32.395135 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:32.395345 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:32.398057 (+ 2712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:32.398064 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:32.398084 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:32.398128 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:32.400485 (+ 2357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:32.401118 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:32.401125 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:32.401132 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:32.401281 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:32.401735 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:32.401738 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:32.401740 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:32.402303 (+ 563us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:32.402515 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:32.402956 (+ 441us) server_negotiation.cc:299] Negotiation successful
0806 11:36:32.403297 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":153,"threads_started":1}
I20260806 11:36:32.407193 25414 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4421473eff604e4f93fa93547e61b2c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4421473eff604e4f93fa93547e61b2c0" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 43645 } }
I20260806 11:36:32.407442 25414 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4421473eff604e4f93fa93547e61b2c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:32.407488 25414 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4421473eff604e4f93fa93547e61b2c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4421473eff604e4f93fa93547e61b2c0, State: Initialized, Role: FOLLOWER
I20260806 11:36:32.407898 25414 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4421473eff604e4f93fa93547e61b2c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4421473eff604e4f93fa93547e61b2c0" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 43645 } }
I20260806 11:36:32.408027 25414 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4421473eff604e4f93fa93547e61b2c0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 11:36:32.408070 25414 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4421473eff604e4f93fa93547e61b2c0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 11:36:32.408248 25414 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4421473eff604e4f93fa93547e61b2c0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:36:32.409375 25414 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4421473eff604e4f93fa93547e61b2c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4421473eff604e4f93fa93547e61b2c0" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 43645 } }
I20260806 11:36:32.409811 25414 leader_election.cc:302] T 00000000000000000000000000000000 P 4421473eff604e4f93fa93547e61b2c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4421473eff604e4f93fa93547e61b2c0; no voters:
I20260806 11:36:32.410226 25414 leader_election.cc:288] T 00000000000000000000000000000000 P 4421473eff604e4f93fa93547e61b2c0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 11:36:32.410346 25419 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4421473eff604e4f93fa93547e61b2c0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:36:32.410562 25419 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4421473eff604e4f93fa93547e61b2c0 [term 1 LEADER]: Becoming Leader. State: Replica: 4421473eff604e4f93fa93547e61b2c0, State: Running, Role: LEADER
I20260806 11:36:32.411010 25419 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4421473eff604e4f93fa93547e61b2c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4421473eff604e4f93fa93547e61b2c0" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 43645 } }
I20260806 11:36:32.411823 25414 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4421473eff604e4f93fa93547e61b2c0 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 11:36:32.413128 25420 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4421473eff604e4f93fa93547e61b2c0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4421473eff604e4f93fa93547e61b2c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4421473eff604e4f93fa93547e61b2c0" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 43645 } } }
I20260806 11:36:32.413230 25421 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4421473eff604e4f93fa93547e61b2c0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4421473eff604e4f93fa93547e61b2c0. Latest consensus state: current_term: 1 leader_uuid: "4421473eff604e4f93fa93547e61b2c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4421473eff604e4f93fa93547e61b2c0" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 43645 } } }
I20260806 11:36:32.413295 25420 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4421473eff604e4f93fa93547e61b2c0 [sys.catalog]: This master's current role is: LEADER
I20260806 11:36:32.413349 25421 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4421473eff604e4f93fa93547e61b2c0 [sys.catalog]: This master's current role is: LEADER
I20260806 11:36:32.413781 25428 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 11:36:32.418293 25428 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 11:36:32.426352 25428 catalog_manager.cc:1347] Generated new cluster ID: f4e8e8326e6f4c679a2bf003e1b61332
I20260806 11:36:32.426481 25428 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 11:36:32.437959 25428 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 11:36:32.439193 25428 catalog_manager.cc:1504] Loading token signing keys...
I20260806 11:36:32.449445 25428 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4421473eff604e4f93fa93547e61b2c0: Generated new TSK 0
I20260806 11:36:32.450412 25428 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.14.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab.
Entry for principal kudu/127.21.14.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab.
I20260806 11:36:32.491696 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.193:0
--local_ip_for_outbound_sockets=127.21.14.193
--webserver_interface=127.21.14.193
--webserver_port=0
--tserver_master_addrs=127.21.14.254:43645
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab
--principal=kudu/127.21.14.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:34395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:36:32.624245 25440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:36:32.624630 25440 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:36:32.624720 25440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:36:32.629395 25440 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:36:32.629503 25440 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:36:32.629606 25440 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.193
I20260806 11:36:32.636132 25440 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:34395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab
--principal=kudu/127.21.14.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.14.193
--webserver_port=0
--tserver_master_addrs=127.21.14.254:43645
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25440
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.193
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:36:32.637480 25440 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:36:32.638661 25440 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:36:32.649948 25445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:32.649948 25448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:32.650009 25446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:32.650554 25440 server_base.cc:1034] running on GCE node
I20260806 11:36:32.651104 25440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:36:32.651958 25440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:36:32.653210 25440 hybrid_clock.cc:648] HybridClock initialized: now 1786016192653161 us; error 67 us; skew 500 ppm
Aug 06 11:36:32 dist-test-slave-7c35 krb5kdc[25329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016192, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:36:32.656804 25440 init.cc:377] Logged in from keytab as kudu/127.21.14.193@KRBTEST.COM (short username kudu)
I20260806 11:36:32.658154 25440 webserver.cc:460] Webserver started at http://127.21.14.193:42117/ using document root <none> and password file <none>
I20260806 11:36:32.658987 25440 fs_manager.cc:359] Metadata directory not provided
I20260806 11:36:32.659121 25440 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:36:32.659384 25440 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:36:32.662036 25440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/minicluster-data/ts-0/data/instance:
uuid: "dda1d588f48e4027aa22c55c9861c5e5"
format_stamp: "Formatted at 2026-08-06 11:36:32 on dist-test-slave-7c35"
server_key: "7559b7a9b4adece7ca76472bace649d1"
server_key_iv: "63f2b9233998ee99f11a46c176631f98"
server_key_version: "encryptionkey@0"
I20260806 11:36:32.662724 25440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/minicluster-data/ts-0/wal/instance:
uuid: "dda1d588f48e4027aa22c55c9861c5e5"
format_stamp: "Formatted at 2026-08-06 11:36:32 on dist-test-slave-7c35"
server_key: "7559b7a9b4adece7ca76472bace649d1"
server_key_iv: "63f2b9233998ee99f11a46c176631f98"
server_key_version: "encryptionkey@0"
I20260806 11:36:32.667411 25440 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 11:36:32.670403 25457 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:32.671990 25440 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 11:36:32.672138 25440 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/minicluster-data/ts-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/minicluster-data/ts-0/wal
uuid: "dda1d588f48e4027aa22c55c9861c5e5"
format_stamp: "Formatted at 2026-08-06 11:36:32 on dist-test-slave-7c35"
server_key: "7559b7a9b4adece7ca76472bace649d1"
server_key_iv: "63f2b9233998ee99f11a46c176631f98"
server_key_version: "encryptionkey@0"
I20260806 11:36:32.672281 25440 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:36:32.691394 25440 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:36:32.695734 25440 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:36:32.695994 25440 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:36:32.696693 25440 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:36:32.697626 25440 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:36:32.697676 25440 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:32.697743 25440 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:36:32.697775 25440 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:32.708523 25440 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.193:39453
I20260806 11:36:32.708566 25577 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.193:39453 every 8 connection(s)
I20260806 11:36:32.709561 25440 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
I20260806 11:36:32.711166 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 25440
I20260806 11:36:32.711330 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/minicluster-data/ts-0/wal/instance
I20260806 11:36:32.711689 21563 external_mini_cluster.cc:1428] Setting key 5f739d839e87c6cde05c6d0186cc63fb
Aug 06 11:36:32 dist-test-slave-7c35 krb5kdc[25329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016192, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.193@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.14.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.194@KRBTEST.COM" created.
I20260806 11:36:32.727023 25417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:32.711986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:39867 (local address 127.21.14.254:43645)
0806 11:36:32.712161 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:32.712166 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:32.713470 (+ 1304us) server_negotiation.cc:407] Connection header received
0806 11:36:32.715195 (+ 1725us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:32.715199 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:32.715259 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:32.715344 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:32.716692 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:32.717264 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:32.718902 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:32.719183 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:32.719324 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:32.722632 (+ 3308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:32.722635 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:32.722637 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:32.722669 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:32.724462 (+ 1793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:32.725094 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:32.725098 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:32.725099 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:32.725185 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:32.725620 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:32.725623 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:32.725625 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:32.725858 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:32.725976 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:32.726622 (+ 646us) server_negotiation.cc:299] Negotiation successful
0806 11:36:32.726782 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 11:36:32.727983 25580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:32.712582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:43645 (local address 127.21.14.193:39867)
0806 11:36:32.713286 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:32.713371 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:32.714951 (+ 1580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:32.715561 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:32.715570 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:32.715973 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:32.716524 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:32.716540 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:32.717434 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:32.717437 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:32.718670 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:32.718684 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:32.719402 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:32.719407 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:32.719611 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:32.720427 (+ 816us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:32.720446 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:32.722377 (+ 1931us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:32.724609 (+ 2232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:32.724617 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:32.724638 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:32.724962 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:32.725287 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:32.725290 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:32.725293 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:32.725506 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:32.725977 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:32.725987 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:32.726347 (+ 360us) client_negotiation.cc:769] Sending connection context
0806 11:36:32.726579 (+ 232us) client_negotiation.cc:240] Negotiation successful
0806 11:36:32.727006 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":229,"threads_started":1}
I20260806 11:36:32.729586 25578 heartbeater.cc:347] Connected to a master server at 127.21.14.254:43645
I20260806 11:36:32.729876 25578 heartbeater.cc:464] Registering TS with master...
I20260806 11:36:32.730492 25578 heartbeater.cc:511] Master 127.21.14.254:43645 requested a full tablet report, sending...
I20260806 11:36:32.732358 25375 ts_manager.cc:194] Registered new tserver with Master: dda1d588f48e4027aa22c55c9861c5e5 (127.21.14.193:39453)
I20260806 11:36:32.733625 25375 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.14.193@KRBTEST.COM'} at 127.21.14.193:39867
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab.
Entry for principal kudu/127.21.14.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab.
I20260806 11:36:32.751658 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.194:0
--local_ip_for_outbound_sockets=127.21.14.194
--webserver_interface=127.21.14.194
--webserver_port=0
--tserver_master_addrs=127.21.14.254:43645
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab
--principal=kudu/127.21.14.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:34395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:36:32.876632 25583 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:36:32.877082 25583 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:36:32.877170 25583 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:36:32.881650 25583 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:36:32.881727 25583 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:36:32.881793 25583 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.194
I20260806 11:36:32.887980 25583 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:34395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab
--principal=kudu/127.21.14.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.14.194
--webserver_port=0
--tserver_master_addrs=127.21.14.254:43645
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25583
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.194
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:36:32.889305 25583 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:36:32.890431 25583 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:36:32.901062 25589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:32.901090 25592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:32.901062 25590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:32.901407 25583 server_base.cc:1034] running on GCE node
I20260806 11:36:32.901844 25583 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:36:32.902518 25583 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:36:32.903721 25583 hybrid_clock.cc:648] HybridClock initialized: now 1786016192903695 us; error 39 us; skew 500 ppm
Aug 06 11:36:32 dist-test-slave-7c35 krb5kdc[25329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016192, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:36:32.906944 25583 init.cc:377] Logged in from keytab as kudu/127.21.14.194@KRBTEST.COM (short username kudu)
I20260806 11:36:32.908001 25583 webserver.cc:460] Webserver started at http://127.21.14.194:33461/ using document root <none> and password file <none>
I20260806 11:36:32.908670 25583 fs_manager.cc:359] Metadata directory not provided
I20260806 11:36:32.908741 25583 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:36:32.908993 25583 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:36:32.911459 25583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/minicluster-data/ts-1/data/instance:
uuid: "c57c225224dc444ba65cfafc2db96b5b"
format_stamp: "Formatted at 2026-08-06 11:36:32 on dist-test-slave-7c35"
server_key: "a188837a4022bcaf86d3c5e308472535"
server_key_iv: "7c1d8d9b8cca16699e40e455964ca811"
server_key_version: "encryptionkey@0"
I20260806 11:36:32.912191 25583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/minicluster-data/ts-1/wal/instance:
uuid: "c57c225224dc444ba65cfafc2db96b5b"
format_stamp: "Formatted at 2026-08-06 11:36:32 on dist-test-slave-7c35"
server_key: "a188837a4022bcaf86d3c5e308472535"
server_key_iv: "7c1d8d9b8cca16699e40e455964ca811"
server_key_version: "encryptionkey@0"
I20260806 11:36:32.916721 25583 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 11:36:32.919621 25599 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:32.921044 25583 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 11:36:32.921202 25583 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/minicluster-data/ts-1/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/minicluster-data/ts-1/wal
uuid: "c57c225224dc444ba65cfafc2db96b5b"
format_stamp: "Formatted at 2026-08-06 11:36:32 on dist-test-slave-7c35"
server_key: "a188837a4022bcaf86d3c5e308472535"
server_key_iv: "7c1d8d9b8cca16699e40e455964ca811"
server_key_version: "encryptionkey@0"
I20260806 11:36:32.921365 25583 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:36:32.944942 25583 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:36:32.948700 25583 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:36:32.948938 25583 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:36:32.949683 25583 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:36:32.950598 25583 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:36:32.950651 25583 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:32.950695 25583 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:36:32.950709 25583 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:32.960758 25583 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.194:36475
I20260806 11:36:32.960839 25716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.194:36475 every 8 connection(s)
I20260806 11:36:32.961871 25583 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
I20260806 11:36:32.970466 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 25583
I20260806 11:36:32.970625 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/minicluster-data/ts-1/wal/instance
I20260806 11:36:32.971035 21563 external_mini_cluster.cc:1428] Setting key 8ba2a9506a089685acf9efc9226d0f1f
Aug 06 11:36:32 dist-test-slave-7c35 krb5kdc[25329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016192, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.194@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:36:32.978408 25417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:32.964246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:54091 (local address 127.21.14.254:43645)
0806 11:36:32.964426 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:32.964432 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:32.965410 (+ 978us) server_negotiation.cc:407] Connection header received
0806 11:36:32.966914 (+ 1504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:32.966918 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:32.966990 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:32.967079 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:32.968482 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:32.969056 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:32.970356 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:32.970598 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:32.970766 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:32.974139 (+ 3373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:32.974143 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:32.974146 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:32.974178 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:32.975890 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:32.976521 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:32.976525 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:32.976526 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:32.976621 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:32.977028 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:32.977032 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:32.977034 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:32.977322 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:32.977448 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:32.978078 (+ 630us) server_negotiation.cc:299] Negotiation successful
0806 11:36:32.978224 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 11:36:32.979427 25719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:32.964710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:43645 (local address 127.21.14.194:54091)
0806 11:36:32.965231 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:32.965280 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:32.966629 (+ 1349us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:32.967299 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:32.967308 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:32.967753 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:32.968302 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:32.968317 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:32.969184 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:32.969186 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:32.970189 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:32.970203 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:32.970894 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:32.970901 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:32.971163 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:32.972121 (+ 958us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:32.972140 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:32.973939 (+ 1799us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:32.976024 (+ 2085us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:32.976053 (+ 29us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:32.976078 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:32.976402 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:32.976719 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:32.976722 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:32.976724 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:32.976924 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:32.977449 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:32.977469 (+ 20us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:32.977773 (+ 304us) client_negotiation.cc:769] Sending connection context
0806 11:36:32.978030 (+ 257us) client_negotiation.cc:240] Negotiation successful
0806 11:36:32.978417 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":135,"threads_started":1}
I20260806 11:36:32.980763 25717 heartbeater.cc:347] Connected to a master server at 127.21.14.254:43645
I20260806 11:36:32.981091 25717 heartbeater.cc:464] Registering TS with master...
I20260806 11:36:32.981832 25717 heartbeater.cc:511] Master 127.21.14.254:43645 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.21.14.195@KRBTEST.COM; defaulting to no policy
I20260806 11:36:32.983279 25375 ts_manager.cc:194] Registered new tserver with Master: c57c225224dc444ba65cfafc2db96b5b (127.21.14.194:36475)
I20260806 11:36:32.983956 25375 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.14.194@KRBTEST.COM'} at 127.21.14.194:54091
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab.
Entry for principal kudu/127.21.14.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab.
I20260806 11:36:33.006654 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.195:0
--local_ip_for_outbound_sockets=127.21.14.195
--webserver_interface=127.21.14.195
--webserver_port=0
--tserver_master_addrs=127.21.14.254:43645
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab
--principal=kudu/127.21.14.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:34395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:36:33.134280 25722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:36:33.134595 25722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:36:33.134644 25722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:36:33.139159 25722 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:36:33.139247 25722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:36:33.139312 25722 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.195
I20260806 11:36:33.145314 25722 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:34395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab
--principal=kudu/127.21.14.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.14.195
--webserver_port=0
--tserver_master_addrs=127.21.14.254:43645
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25722
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.195
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:36:33.146606 25722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:36:33.147876 25722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:36:33.160377 25728 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:33.160279 25730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:33.160420 25722 server_base.cc:1034] running on GCE node
W20260806 11:36:33.160511 25727 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:33.161254 25722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:36:33.162083 25722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:36:33.163368 25722 hybrid_clock.cc:648] HybridClock initialized: now 1786016193163324 us; error 69 us; skew 500 ppm
Aug 06 11:36:33 dist-test-slave-7c35 krb5kdc[25329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016193, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:36:33.166960 25722 init.cc:377] Logged in from keytab as kudu/127.21.14.195@KRBTEST.COM (short username kudu)
I20260806 11:36:33.168156 25722 webserver.cc:460] Webserver started at http://127.21.14.195:36321/ using document root <none> and password file <none>
I20260806 11:36:33.168838 25722 fs_manager.cc:359] Metadata directory not provided
I20260806 11:36:33.168949 25722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:36:33.169227 25722 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:36:33.171867 25722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/minicluster-data/ts-2/data/instance:
uuid: "a302a623d15e414992a07eb778c1af88"
format_stamp: "Formatted at 2026-08-06 11:36:33 on dist-test-slave-7c35"
server_key: "a4f08c2d1469e77da50929ddf168908e"
server_key_iv: "7ce246dc06edb806d7181e825f7e119c"
server_key_version: "encryptionkey@0"
I20260806 11:36:33.172614 25722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/minicluster-data/ts-2/wal/instance:
uuid: "a302a623d15e414992a07eb778c1af88"
format_stamp: "Formatted at 2026-08-06 11:36:33 on dist-test-slave-7c35"
server_key: "a4f08c2d1469e77da50929ddf168908e"
server_key_iv: "7ce246dc06edb806d7181e825f7e119c"
server_key_version: "encryptionkey@0"
I20260806 11:36:33.177837 25722 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 11:36:33.181444 25738 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:33.183547 25722 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260806 11:36:33.183756 25722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/minicluster-data/ts-2/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/minicluster-data/ts-2/wal
uuid: "a302a623d15e414992a07eb778c1af88"
format_stamp: "Formatted at 2026-08-06 11:36:33 on dist-test-slave-7c35"
server_key: "a4f08c2d1469e77da50929ddf168908e"
server_key_iv: "7ce246dc06edb806d7181e825f7e119c"
server_key_version: "encryptionkey@0"
I20260806 11:36:33.183918 25722 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:36:33.203219 25722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:36:33.207546 25722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:36:33.207810 25722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:36:33.208529 25722 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:36:33.209493 25722 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:36:33.209570 25722 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:33.209645 25722 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:36:33.209704 25722 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:33.220050 25722 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.195:36323
I20260806 11:36:33.220105 25858 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.195:36323 every 8 connection(s)
I20260806 11:36:33.221127 25722 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
I20260806 11:36:33.226205 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 25722
I20260806 11:36:33.226382 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786016169045816-21563-0/minicluster-data/ts-2/wal/instance
I20260806 11:36:33.226661 21563 external_mini_cluster.cc:1428] Setting key 8edaa6073e43cd578f2303f7db42baa4
Aug 06 11:36:33 dist-test-slave-7c35 krb5kdc[25329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016193, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.195@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:36:33.237105 25417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:33.223317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:50057 (local address 127.21.14.254:43645)
0806 11:36:33.223502 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:33.223507 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:33.224423 (+ 916us) server_negotiation.cc:407] Connection header received
0806 11:36:33.226100 (+ 1677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:33.226103 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:33.226158 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:33.226295 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:33.227988 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:33.228682 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:33.230139 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:33.230417 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:33.230525 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:33.233365 (+ 2840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:33.233369 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:33.233371 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:33.233426 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:33.234835 (+ 1409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:33.235462 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:33.235465 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:33.235466 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:33.235550 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:33.235932 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:33.235935 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:33.235937 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:33.236161 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:33.236252 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:33.236829 (+ 577us) server_negotiation.cc:299] Negotiation successful
0806 11:36:33.236959 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260806 11:36:33.238059 25861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:33.223653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:43645 (local address 127.21.14.195:50057)
0806 11:36:33.224197 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:33.224245 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:33.225780 (+ 1535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:33.226523 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:33.226532 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:33.227016 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:33.227832 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:33.227852 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:33.228838 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:33.228841 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:33.229973 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:33.229985 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:33.230610 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:33.230614 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:33.230778 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:33.231561 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:33.231580 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:33.233155 (+ 1575us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:33.235052 (+ 1897us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:33.235057 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:33.235074 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:33.235362 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:33.235641 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:33.235644 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:33.235645 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:33.235805 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:33.236276 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:33.236282 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:33.236576 (+ 294us) client_negotiation.cc:769] Sending connection context
0806 11:36:33.236755 (+ 179us) client_negotiation.cc:240] Negotiation successful
0806 11:36:33.237077 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":135,"threads_started":1}
I20260806 11:36:33.239462 25859 heartbeater.cc:347] Connected to a master server at 127.21.14.254:43645
I20260806 11:36:33.239768 25859 heartbeater.cc:464] Registering TS with master...
I20260806 11:36:33.240528 25859 heartbeater.cc:511] Master 127.21.14.254:43645 requested a full tablet report, sending...
I20260806 11:36:33.241726 25375 ts_manager.cc:194] Registered new tserver with Master: a302a623d15e414992a07eb778c1af88 (127.21.14.195:36323)
I20260806 11:36:33.242290 25375 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.14.195@KRBTEST.COM'} at 127.21.14.195:50057
I20260806 11:36:33.253479 21563 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 11:36:33 dist-test-slave-7c35 krb5kdc[25329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016193, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 11:36:33 dist-test-slave-7c35 krb5kdc[25329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016193, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.14.193@KRBTEST.COM
I20260806 11:36:33.284430 25866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:33.273313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46868 (local address 127.21.14.193:39453)
0806 11:36:33.273646 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:33.273653 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:33.273682 (+ 29us) server_negotiation.cc:407] Connection header received
0806 11:36:33.273737 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:33.273743 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:33.273951 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:33.274082 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:33.274975 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:33.275833 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:33.276873 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:33.277370 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:33.277476 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:33.280261 (+ 2785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:33.280276 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:33.280293 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:33.280325 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:33.282281 (+ 1956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:33.282824 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:33.282830 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:33.282832 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:33.282998 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:33.283287 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:33.283290 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:33.283291 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:33.283619 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:33.283793 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:33.284013 (+ 220us) server_negotiation.cc:299] Negotiation successful
0806 11:36:33.284197 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":92,"threads_started":1}
Aug 06 11:36:33 dist-test-slave-7c35 krb5kdc[25329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016193, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:36:33.297902 25417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:33.287776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46088 (local address 127.21.14.254:43645)
0806 11:36:33.287957 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:33.287961 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:33.287999 (+ 38us) server_negotiation.cc:407] Connection header received
0806 11:36:33.288208 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:33.288211 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:33.288259 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:33.288333 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:33.289139 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:33.289817 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:33.290818 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:33.291151 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:33.291288 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:33.294128 (+ 2840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:33.294132 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:33.294134 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:33.294169 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:33.295881 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:33.296414 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:33.296419 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:33.296428 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:33.296519 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:33.296843 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:33.296846 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:33.296848 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:33.297120 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:33.297280 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:33.297575 (+ 295us) server_negotiation.cc:299] Negotiation successful
0806 11:36:33.297711 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260806 11:36:33.298422 25375 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46088
I20260806 11:36:33.299573 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 25440
I20260806 11:36:33.308694 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 25583
I20260806 11:36:33.316705 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 25722
I20260806 11:36:33.324717 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 25342
2026-08-06T11:36:33Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2416 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 11:36:33 dist-test-slave-7c35 krb5kdc[25871](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 11:36:33 dist-test-slave-7c35 krb5kdc[25871](info): set up 2 sockets
Aug 06 11:36:33 dist-test-slave-7c35 krb5kdc[25871](info): commencing operation
krb5kdc: starting...
W20260806 11:36:34.417935 21563 mini_kdc.cc:121] Time spent starting KDC: real 1.067s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Aug 06 11:36:34 dist-test-slave-7c35 krb5kdc[25871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016194, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T11:36:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T11:36:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.14.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab.
Entry for principal kudu/127.21.14.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab.
I20260806 11:36:34.566620 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:45651
--webserver_interface=127.21.14.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab
--principal=kudu/127.21.14.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:38223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.14.254:45651
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:36:34.694998 25886 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:36:34.695314 25886 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:36:34.695360 25886 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:36:34.700106 25886 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 11:36:34.700206 25886 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:36:34.700227 25886 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:36:34.700244 25886 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 11:36:34.700260 25886 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 11:36:34.706157 25886 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:38223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.14.254:45651
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab
--principal=kudu/127.21.14.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:45651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.14.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25886
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:36:34.707533 25886 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:36:34.708717 25886 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:36:34.718000 25891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:34.718029 25894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:34.718066 25886 server_base.cc:1034] running on GCE node
W20260806 11:36:34.718000 25892 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:34.719087 25886 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:36:34.720397 25886 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:36:34.721638 25886 hybrid_clock.cc:648] HybridClock initialized: now 1786016194721622 us; error 32 us; skew 500 ppm
Aug 06 11:36:34 dist-test-slave-7c35 krb5kdc[25871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016194, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:36:34.729223 25886 init.cc:377] Logged in from keytab as kudu/127.21.14.254@KRBTEST.COM (short username kudu)
I20260806 11:36:34.730230 25886 fs_manager.cc:359] Metadata directory not provided
I20260806 11:36:34.730358 25886 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:36:34.730686 25886 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:36:34.733263 25886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/minicluster-data/master-0/data/instance:
uuid: "1ada5e0a765d49a38476b82c8653405d"
format_stamp: "Formatted at 2026-08-06 11:36:34 on dist-test-slave-7c35"
server_key: "c02d25af8d2168d7d51f65d6f1629778"
server_key_iv: "a8d2805edee7c79a79f02bb8383b8a67"
server_key_version: "encryptionkey@0"
I20260806 11:36:34.733960 25886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/minicluster-data/master-0/wal/instance:
uuid: "1ada5e0a765d49a38476b82c8653405d"
format_stamp: "Formatted at 2026-08-06 11:36:34 on dist-test-slave-7c35"
server_key: "c02d25af8d2168d7d51f65d6f1629778"
server_key_iv: "a8d2805edee7c79a79f02bb8383b8a67"
server_key_version: "encryptionkey@0"
I20260806 11:36:34.739413 25886 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 11:36:34.742719 25900 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:34.744534 25886 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 11:36:34.744755 25886 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/minicluster-data/master-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/minicluster-data/master-0/wal
uuid: "1ada5e0a765d49a38476b82c8653405d"
format_stamp: "Formatted at 2026-08-06 11:36:34 on dist-test-slave-7c35"
server_key: "c02d25af8d2168d7d51f65d6f1629778"
server_key_iv: "a8d2805edee7c79a79f02bb8383b8a67"
server_key_version: "encryptionkey@0"
I20260806 11:36:34.745067 25886 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:36:34.768096 25886 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:36:34.772101 25886 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:36:34.772451 25886 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:36:34.780576 25886 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.254:45651
I20260806 11:36:34.780907 25955 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.254:45651 every 8 connection(s)
I20260806 11:36:34.781368 25886 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
I20260806 11:36:34.784588 25956 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:34.785915 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 25886
I20260806 11:36:34.786067 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/minicluster-data/master-0/wal/instance
I20260806 11:36:34.786360 21563 external_mini_cluster.cc:1428] Setting key ea070f85a70b42fdff354ffcdb48bd52
I20260806 11:36:34.791857 25956 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ada5e0a765d49a38476b82c8653405d: Bootstrap starting.
I20260806 11:36:34.794991 25956 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ada5e0a765d49a38476b82c8653405d: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:34.796200 25956 log.cc:824] T 00000000000000000000000000000000 P 1ada5e0a765d49a38476b82c8653405d: Log is configured to *not* fsync() on all Append() calls
Aug 06 11:36:34 dist-test-slave-7c35 krb5kdc[25871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016194, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:36:34.799172 25956 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ada5e0a765d49a38476b82c8653405d: No bootstrap required, opened a new log
I20260806 11:36:34.803112 25959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:34.788082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53930 (local address 127.21.14.254:45651)
0806 11:36:34.788631 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:34.788662 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:34.788705 (+ 43us) server_negotiation.cc:407] Connection header received
0806 11:36:34.789415 (+ 710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:34.789435 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:34.789829 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:34.790166 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:34.791273 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:34.792413 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:34.793688 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:34.794229 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:34.794374 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:34.797671 (+ 3297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:34.797677 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:34.797693 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:34.797724 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:34.799852 (+ 2128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:34.800381 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:34.800387 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:34.800392 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:34.800524 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:34.800836 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:34.800839 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:34.800841 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:34.801295 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:34.801526 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:34.801838 (+ 312us) server_negotiation.cc:299] Negotiation successful
0806 11:36:34.802182 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":181,"threads_started":1}
I20260806 11:36:34.803776 25956 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ada5e0a765d49a38476b82c8653405d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ada5e0a765d49a38476b82c8653405d" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 45651 } }
I20260806 11:36:34.804060 25956 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1ada5e0a765d49a38476b82c8653405d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:34.804145 25956 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ada5e0a765d49a38476b82c8653405d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ada5e0a765d49a38476b82c8653405d, State: Initialized, Role: FOLLOWER
I20260806 11:36:34.804724 25956 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ada5e0a765d49a38476b82c8653405d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ada5e0a765d49a38476b82c8653405d" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 45651 } }
I20260806 11:36:34.804859 25956 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ada5e0a765d49a38476b82c8653405d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 11:36:34.804941 25956 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ada5e0a765d49a38476b82c8653405d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 11:36:34.805051 25956 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ada5e0a765d49a38476b82c8653405d [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:36:34.806238 25956 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ada5e0a765d49a38476b82c8653405d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ada5e0a765d49a38476b82c8653405d" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 45651 } }
I20260806 11:36:34.806640 25956 leader_election.cc:302] T 00000000000000000000000000000000 P 1ada5e0a765d49a38476b82c8653405d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ada5e0a765d49a38476b82c8653405d; no voters:
I20260806 11:36:34.807080 25956 leader_election.cc:288] T 00000000000000000000000000000000 P 1ada5e0a765d49a38476b82c8653405d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 11:36:34.807332 25961 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ada5e0a765d49a38476b82c8653405d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:36:34.807615 25961 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ada5e0a765d49a38476b82c8653405d [term 1 LEADER]: Becoming Leader. State: Replica: 1ada5e0a765d49a38476b82c8653405d, State: Running, Role: LEADER
I20260806 11:36:34.808097 25961 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ada5e0a765d49a38476b82c8653405d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ada5e0a765d49a38476b82c8653405d" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 45651 } }
I20260806 11:36:34.808771 25956 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ada5e0a765d49a38476b82c8653405d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 11:36:34.809391 25961 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ada5e0a765d49a38476b82c8653405d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ada5e0a765d49a38476b82c8653405d. Latest consensus state: current_term: 1 leader_uuid: "1ada5e0a765d49a38476b82c8653405d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ada5e0a765d49a38476b82c8653405d" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 45651 } } }
I20260806 11:36:34.809557 25961 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ada5e0a765d49a38476b82c8653405d [sys.catalog]: This master's current role is: LEADER
I20260806 11:36:34.810041 25967 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 11:36:34.809993 25962 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ada5e0a765d49a38476b82c8653405d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1ada5e0a765d49a38476b82c8653405d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ada5e0a765d49a38476b82c8653405d" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 45651 } } }
I20260806 11:36:34.810163 25962 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ada5e0a765d49a38476b82c8653405d [sys.catalog]: This master's current role is: LEADER
I20260806 11:36:34.813761 25967 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 11:36:34.820992 25967 catalog_manager.cc:1347] Generated new cluster ID: 88b2b93e5d1e4f47abc600187634964d
I20260806 11:36:34.821108 25967 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 11:36:34.826972 25967 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 11:36:34.828177 25967 catalog_manager.cc:1504] Loading token signing keys...
I20260806 11:36:34.835758 25967 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1ada5e0a765d49a38476b82c8653405d: Generated new TSK 0
I20260806 11:36:34.836707 25967 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.14.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab.
Entry for principal kudu/127.21.14.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab.
I20260806 11:36:34.882789 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.193:0
--local_ip_for_outbound_sockets=127.21.14.193
--webserver_interface=127.21.14.193
--webserver_port=0
--tserver_master_addrs=127.21.14.254:45651
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab
--principal=kudu/127.21.14.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:38223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:36:35.025957 25982 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:36:35.026258 25982 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:36:35.026305 25982 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:36:35.030377 25982 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:36:35.030443 25982 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:36:35.030510 25982 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.193
I20260806 11:36:35.035926 25982 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:38223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab
--principal=kudu/127.21.14.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.14.193
--webserver_port=0
--tserver_master_addrs=127.21.14.254:45651
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25982
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.193
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:36:35.037039 25982 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:36:35.038051 25982 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:36:35.048432 25988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:35.048632 25982 server_base.cc:1034] running on GCE node
W20260806 11:36:35.048446 25990 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:35.048432 25987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:35.049219 25982 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:36:35.049998 25982 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:36:35.051283 25982 hybrid_clock.cc:648] HybridClock initialized: now 1786016195051264 us; error 31 us; skew 500 ppm
Aug 06 11:36:35 dist-test-slave-7c35 krb5kdc[25871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016195, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:36:35.054320 25982 init.cc:377] Logged in from keytab as kudu/127.21.14.193@KRBTEST.COM (short username kudu)
I20260806 11:36:35.055229 25982 fs_manager.cc:359] Metadata directory not provided
I20260806 11:36:35.055303 25982 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:36:35.055498 25982 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:36:35.057729 25982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/minicluster-data/ts-0/data/instance:
uuid: "8990cb0941cb43c7ae35da135e91ffc5"
format_stamp: "Formatted at 2026-08-06 11:36:35 on dist-test-slave-7c35"
server_key: "d5290ccf8d861008f1b03f5c5f37d2f5"
server_key_iv: "a906d3089c7247593d579716753f75fe"
server_key_version: "encryptionkey@0"
I20260806 11:36:35.058343 25982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/minicluster-data/ts-0/wal/instance:
uuid: "8990cb0941cb43c7ae35da135e91ffc5"
format_stamp: "Formatted at 2026-08-06 11:36:35 on dist-test-slave-7c35"
server_key: "d5290ccf8d861008f1b03f5c5f37d2f5"
server_key_iv: "a906d3089c7247593d579716753f75fe"
server_key_version: "encryptionkey@0"
I20260806 11:36:35.062740 25982 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.007s
I20260806 11:36:35.065613 25996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:35.067312 25982 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 11:36:35.067483 25982 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/minicluster-data/ts-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/minicluster-data/ts-0/wal
uuid: "8990cb0941cb43c7ae35da135e91ffc5"
format_stamp: "Formatted at 2026-08-06 11:36:35 on dist-test-slave-7c35"
server_key: "d5290ccf8d861008f1b03f5c5f37d2f5"
server_key_iv: "a906d3089c7247593d579716753f75fe"
server_key_version: "encryptionkey@0"
I20260806 11:36:35.067629 25982 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:36:35.118096 25982 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:36:35.122165 25982 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:36:35.122490 25982 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:36:35.123361 25982 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:36:35.124576 25982 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:36:35.124632 25982 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:35.124707 25982 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:36:35.124738 25982 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:35.135205 25982 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.193:35357
I20260806 11:36:35.135222 26116 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.193:35357 every 8 connection(s)
I20260806 11:36:35.135927 25982 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
I20260806 11:36:35.143695 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 25982
I20260806 11:36:35.143842 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/minicluster-data/ts-0/wal/instance
I20260806 11:36:35.144191 21563 external_mini_cluster.cc:1428] Setting key ff0326e5a7ac3a22db9a1576751df8df
Aug 06 11:36:35 dist-test-slave-7c35 krb5kdc[25871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016195, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.193@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:36:35.154305 25959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:35.138176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:52123 (local address 127.21.14.254:45651)
0806 11:36:35.138387 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:35.138393 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:35.139431 (+ 1038us) server_negotiation.cc:407] Connection header received
0806 11:36:35.140988 (+ 1557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:35.140992 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:35.141062 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:35.141171 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:35.142660 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:35.143341 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:35.145163 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:35.145553 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:35.145681 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:35.149083 (+ 3402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:35.149090 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:35.149094 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:35.149138 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:35.151337 (+ 2199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:35.152135 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:35.152143 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:35.152147 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:35.152266 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:35.152707 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:35.152714 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:35.152719 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:35.153029 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:35.153157 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:35.153878 (+ 721us) server_negotiation.cc:299] Negotiation successful
0806 11:36:35.154082 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":91}
I20260806 11:36:35.155253 26119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:35.138607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:45651 (local address 127.21.14.193:52123)
0806 11:36:35.139236 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:35.139292 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:35.140709 (+ 1417us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:35.141393 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:35.141402 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:35.141892 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:35.142468 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:35.142484 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:35.143541 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:35.143545 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:35.144988 (+ 1443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:35.145001 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:35.145803 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:35.145812 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:35.146045 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:35.146755 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:35.146784 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:35.148800 (+ 2016us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:35.151583 (+ 2783us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:35.151591 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:35.151610 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:35.151952 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:35.152392 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:35.152394 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:35.152396 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:35.152591 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:35.153252 (+ 661us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:35.153259 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:35.153582 (+ 323us) client_negotiation.cc:769] Sending connection context
0806 11:36:35.153780 (+ 198us) client_negotiation.cc:240] Negotiation successful
0806 11:36:35.154108 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":182,"threads_started":1}
WARNING: no policy specified for kudu/127.21.14.194@KRBTEST.COM; defaulting to no policy
I20260806 11:36:35.156939 26117 heartbeater.cc:347] Connected to a master server at 127.21.14.254:45651
I20260806 11:36:35.157204 26117 heartbeater.cc:464] Registering TS with master...
I20260806 11:36:35.157858 26117 heartbeater.cc:511] Master 127.21.14.254:45651 requested a full tablet report, sending...
I20260806 11:36:35.160009 25918 ts_manager.cc:194] Registered new tserver with Master: 8990cb0941cb43c7ae35da135e91ffc5 (127.21.14.193:35357)
I20260806 11:36:35.161466 25918 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.14.193@KRBTEST.COM'} at 127.21.14.193:52123
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab.
Entry for principal kudu/127.21.14.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab.
I20260806 11:36:35.184417 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.194:0
--local_ip_for_outbound_sockets=127.21.14.194
--webserver_interface=127.21.14.194
--webserver_port=0
--tserver_master_addrs=127.21.14.254:45651
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab
--principal=kudu/127.21.14.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:38223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:36:35.312994 26122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:36:35.313337 26122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:36:35.313390 26122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:36:35.318221 26122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:36:35.318318 26122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:36:35.318388 26122 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.194
I20260806 11:36:35.324213 26122 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:38223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab
--principal=kudu/127.21.14.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.14.194
--webserver_port=0
--tserver_master_addrs=127.21.14.254:45651
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26122
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.194
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:36:35.325392 26122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:36:35.326450 26122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:36:35.337273 26128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:35.337253 26131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:35.337273 26129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:35.337584 26122 server_base.cc:1034] running on GCE node
I20260806 11:36:35.338191 26122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:36:35.339013 26122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:36:35.340243 26122 hybrid_clock.cc:648] HybridClock initialized: now 1786016195340223 us; error 36 us; skew 500 ppm
Aug 06 11:36:35 dist-test-slave-7c35 krb5kdc[25871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016195, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:36:35.343711 26122 init.cc:377] Logged in from keytab as kudu/127.21.14.194@KRBTEST.COM (short username kudu)
I20260806 11:36:35.344597 26122 fs_manager.cc:359] Metadata directory not provided
I20260806 11:36:35.344702 26122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:36:35.344964 26122 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:36:35.347291 26122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/minicluster-data/ts-1/data/instance:
uuid: "7b34cb10ebfd4071b724a0d61188c817"
format_stamp: "Formatted at 2026-08-06 11:36:35 on dist-test-slave-7c35"
server_key: "f6824bcc1f1ac7b43a9f6db4e2eeba77"
server_key_iv: "3f651b9e84786a80e0b3f59c16f951aa"
server_key_version: "encryptionkey@0"
I20260806 11:36:35.347922 26122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/minicluster-data/ts-1/wal/instance:
uuid: "7b34cb10ebfd4071b724a0d61188c817"
format_stamp: "Formatted at 2026-08-06 11:36:35 on dist-test-slave-7c35"
server_key: "f6824bcc1f1ac7b43a9f6db4e2eeba77"
server_key_iv: "3f651b9e84786a80e0b3f59c16f951aa"
server_key_version: "encryptionkey@0"
I20260806 11:36:35.352464 26122 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 11:36:35.355175 26137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:35.356535 26122 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 11:36:35.356694 26122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/minicluster-data/ts-1/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/minicluster-data/ts-1/wal
uuid: "7b34cb10ebfd4071b724a0d61188c817"
format_stamp: "Formatted at 2026-08-06 11:36:35 on dist-test-slave-7c35"
server_key: "f6824bcc1f1ac7b43a9f6db4e2eeba77"
server_key_iv: "3f651b9e84786a80e0b3f59c16f951aa"
server_key_version: "encryptionkey@0"
I20260806 11:36:35.356822 26122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:36:35.372603 26122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:36:35.376834 26122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:36:35.377110 26122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:36:35.378077 26122 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:36:35.379233 26122 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:36:35.379315 26122 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:35.379395 26122 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:36:35.379444 26122 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:35.389183 26122 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.194:45625
I20260806 11:36:35.389204 26254 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.194:45625 every 8 connection(s)
I20260806 11:36:35.389890 26122 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
I20260806 11:36:35.393869 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 26122
I20260806 11:36:35.394017 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/minicluster-data/ts-1/wal/instance
I20260806 11:36:35.394340 21563 external_mini_cluster.cc:1428] Setting key dca861e63530ed9e10b5479ec8c4905d
Aug 06 11:36:35 dist-test-slave-7c35 krb5kdc[25871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016195, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.194@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.14.195@KRBTEST.COM; defaulting to no policy
I20260806 11:36:35.407930 25959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:35.391935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:49613 (local address 127.21.14.254:45651)
0806 11:36:35.392088 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:35.392092 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:35.393612 (+ 1520us) server_negotiation.cc:407] Connection header received
0806 11:36:35.394960 (+ 1348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:35.394966 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:35.395046 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:35.395123 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:35.396824 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:35.397384 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:35.399184 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:35.399600 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:35.399744 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:35.403315 (+ 3571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:35.403322 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:35.403326 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:35.403371 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:35.405289 (+ 1918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:35.405974 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:35.405981 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:35.405984 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:35.406100 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:35.406490 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:35.406502 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:35.406505 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:35.406798 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:35.406983 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:35.407591 (+ 608us) server_negotiation.cc:299] Negotiation successful
0806 11:36:35.407774 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 11:36:35.409296 26257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:35.392357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:45651 (local address 127.21.14.194:49613)
0806 11:36:35.393377 (+ 1020us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:35.393452 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:35.394638 (+ 1186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:35.395362 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:35.395370 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:35.395856 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:35.396627 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:35.396655 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:35.397560 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:35.397567 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:35.399039 (+ 1472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:35.399051 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:35.399792 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:35.399795 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:35.399975 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:35.400967 (+ 992us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:35.400988 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:35.403061 (+ 2073us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:35.405481 (+ 2420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:35.405488 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:35.405504 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:35.405841 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:35.406213 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:35.406215 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:35.406217 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:35.406377 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:35.406994 (+ 617us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:35.407001 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:35.407296 (+ 295us) client_negotiation.cc:769] Sending connection context
0806 11:36:35.407496 (+ 200us) client_negotiation.cc:240] Negotiation successful
0806 11:36:35.407853 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":181,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.195@KRBTEST.COM" created.
I20260806 11:36:35.411062 26255 heartbeater.cc:347] Connected to a master server at 127.21.14.254:45651
I20260806 11:36:35.411358 26255 heartbeater.cc:464] Registering TS with master...
I20260806 11:36:35.411983 26255 heartbeater.cc:511] Master 127.21.14.254:45651 requested a full tablet report, sending...
I20260806 11:36:35.413396 25918 ts_manager.cc:194] Registered new tserver with Master: 7b34cb10ebfd4071b724a0d61188c817 (127.21.14.194:45625)
I20260806 11:36:35.414229 25918 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.14.194@KRBTEST.COM'} at 127.21.14.194:49613
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab.
Entry for principal kudu/127.21.14.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab.
I20260806 11:36:35.432041 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.195:0
--local_ip_for_outbound_sockets=127.21.14.195
--webserver_interface=127.21.14.195
--webserver_port=0
--tserver_master_addrs=127.21.14.254:45651
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab
--principal=kudu/127.21.14.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:38223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:36:35.554831 26260 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:36:35.555171 26260 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:36:35.555217 26260 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:36:35.559355 26260 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:36:35.559422 26260 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:36:35.559485 26260 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.195
I20260806 11:36:35.565377 26260 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:38223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab
--principal=kudu/127.21.14.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.14.195
--webserver_port=0
--tserver_master_addrs=127.21.14.254:45651
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26260
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.195
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:36:35.566533 26260 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:36:35.567745 26260 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:36:35.578908 26270 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:35.579083 26260 server_base.cc:1034] running on GCE node
W20260806 11:36:35.578912 26268 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:35.578908 26267 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:35.579754 26260 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:36:35.580446 26260 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:36:35.581678 26260 hybrid_clock.cc:648] HybridClock initialized: now 1786016195581613 us; error 50 us; skew 500 ppm
Aug 06 11:36:35 dist-test-slave-7c35 krb5kdc[25871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016195, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:36:35.584987 26260 init.cc:377] Logged in from keytab as kudu/127.21.14.195@KRBTEST.COM (short username kudu)
I20260806 11:36:35.585794 26260 fs_manager.cc:359] Metadata directory not provided
I20260806 11:36:35.585863 26260 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:36:35.586055 26260 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:36:35.588335 26260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/minicluster-data/ts-2/data/instance:
uuid: "da9c1362b3864546ac2207e9ed2606bb"
format_stamp: "Formatted at 2026-08-06 11:36:35 on dist-test-slave-7c35"
server_key: "855f90d957727c63a32770b3bc9e5361"
server_key_iv: "c53a7ce62809f560c73592b79b468efa"
server_key_version: "encryptionkey@0"
I20260806 11:36:35.588932 26260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/minicluster-data/ts-2/wal/instance:
uuid: "da9c1362b3864546ac2207e9ed2606bb"
format_stamp: "Formatted at 2026-08-06 11:36:35 on dist-test-slave-7c35"
server_key: "855f90d957727c63a32770b3bc9e5361"
server_key_iv: "c53a7ce62809f560c73592b79b468efa"
server_key_version: "encryptionkey@0"
I20260806 11:36:35.593755 26260 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 11:36:35.597005 26277 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:35.598729 26260 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 11:36:35.598953 26260 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/minicluster-data/ts-2/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/minicluster-data/ts-2/wal
uuid: "da9c1362b3864546ac2207e9ed2606bb"
format_stamp: "Formatted at 2026-08-06 11:36:35 on dist-test-slave-7c35"
server_key: "855f90d957727c63a32770b3bc9e5361"
server_key_iv: "c53a7ce62809f560c73592b79b468efa"
server_key_version: "encryptionkey@0"
I20260806 11:36:35.599104 26260 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:36:35.623291 26260 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:36:35.628289 26260 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:36:35.628564 26260 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:36:35.629314 26260 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:36:35.630566 26260 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:36:35.630643 26260 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:35.630731 26260 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:36:35.630776 26260 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:35.641707 26260 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.195:40495
I20260806 11:36:35.641733 26399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.195:40495 every 8 connection(s)
I20260806 11:36:35.642549 26260 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
I20260806 11:36:35.651261 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 26260
I20260806 11:36:35.651484 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/minicluster-data/ts-2/wal/instance
I20260806 11:36:35.651870 21563 external_mini_cluster.cc:1428] Setting key af75baf37d585649890d5a9996b4794b
Aug 06 11:36:35 dist-test-slave-7c35 krb5kdc[25871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016195, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.195@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:36:35.659669 25959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:35.645244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:37157 (local address 127.21.14.254:45651)
0806 11:36:35.645418 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:35.645423 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:35.646974 (+ 1551us) server_negotiation.cc:407] Connection header received
0806 11:36:35.647987 (+ 1013us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:35.647991 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:35.648054 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:35.648165 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:35.649668 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:35.650268 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:35.651874 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:35.652266 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:35.652403 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:35.655552 (+ 3149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:35.655556 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:35.655558 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:35.655589 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:35.657132 (+ 1543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:35.657781 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:35.657784 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:35.657786 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:35.657880 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:35.658341 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:35.658344 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:35.658346 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:35.658571 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:35.658658 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:35.659383 (+ 725us) server_negotiation.cc:299] Negotiation successful
0806 11:36:35.659524 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 11:36:35.660655 26402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:35.645694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:45651 (local address 127.21.14.195:37157)
0806 11:36:35.646718 (+ 1024us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:35.646768 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:35.647707 (+ 939us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:35.648356 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:35.648364 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:35.648805 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:35.649454 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:35.649471 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:35.650449 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:35.650452 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:35.651715 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:35.651727 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:35.652508 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:35.652514 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:35.652690 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:35.653478 (+ 788us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:35.653502 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:35.655324 (+ 1822us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:35.657274 (+ 1950us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:35.657280 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:35.657297 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:35.657637 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:35.658010 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:35.658013 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:35.658015 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:35.658186 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:35.658699 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:35.658706 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:35.659090 (+ 384us) client_negotiation.cc:769] Sending connection context
0806 11:36:35.659280 (+ 190us) client_negotiation.cc:240] Negotiation successful
0806 11:36:35.659617 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":168,"threads_started":1}
I20260806 11:36:35.662163 26400 heartbeater.cc:347] Connected to a master server at 127.21.14.254:45651
I20260806 11:36:35.662410 26400 heartbeater.cc:464] Registering TS with master...
I20260806 11:36:35.662971 26400 heartbeater.cc:511] Master 127.21.14.254:45651 requested a full tablet report, sending...
I20260806 11:36:35.664093 25918 ts_manager.cc:194] Registered new tserver with Master: da9c1362b3864546ac2207e9ed2606bb (127.21.14.195:40495)
I20260806 11:36:35.664666 25918 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.14.195@KRBTEST.COM'} at 127.21.14.195:37157
I20260806 11:36:35.666486 21563 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 11:36:35.676729 25959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:35.668662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53932 (local address 127.21.14.254:45651)
0806 11:36:35.668772 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:35.668777 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:35.669049 (+ 272us) server_negotiation.cc:407] Connection header received
0806 11:36:35.669147 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:35.669149 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:35.669190 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:35.669261 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:35.669983 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:35.670511 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:35.671460 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:35.671684 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:35.671806 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:35.673240 (+ 1434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:35.673243 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:35.673246 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:35.673272 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:35.674837 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:35.675348 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:35.675353 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:35.675356 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:35.675466 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:35.675755 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:35.675757 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:35.675759 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:35.676098 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:35.676239 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:35.676467 (+ 228us) server_negotiation.cc:299] Negotiation successful
0806 11:36:35.676587 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 11:36:35.681116 25918 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53932:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 11:36:35.684975 25918 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 11:36:35.702204 26411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:35.695806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.194:45625 (local address 127.0.0.1:45408)
0806 11:36:35.696209 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:35.696271 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:35.696421 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:35.697278 (+ 857us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:35.697284 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:35.697309 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:36:35.697461 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:35.697468 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:35.699035 (+ 1567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:35.699042 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:35.700609 (+ 1567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:35.700620 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:35.701530 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:35.701534 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:35.701668 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:35.701708 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 11:36:35.701830 (+ 122us) client_negotiation.cc:240] Negotiation successful
0806 11:36:35.702024 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"mutex_wait_us":96,"thread_start_us":106,"threads_started":1}
I20260806 11:36:35.702204 26412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:35.695965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45408 (local address 127.21.14.194:45625)
0806 11:36:35.696497 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:35.696525 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:35.696552 (+ 27us) server_negotiation.cc:407] Connection header received
0806 11:36:35.696639 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:35.696645 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:35.696867 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:35.697049 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:36:35.697619 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:35.698884 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:35.700753 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:35.701377 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:35.701524 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:35.701710 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:35.701874 (+ 164us) server_negotiation.cc:299] Negotiation successful
0806 11:36:35.701998 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":180,"threads_started":1}
I20260806 11:36:35.702275 26413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:35.696654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.195:40495 (local address 127.0.0.1:59078)
0806 11:36:35.696888 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:35.696904 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:35.697026 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:35.697823 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:35.697826 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:35.697839 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:36:35.697976 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:35.697981 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:35.699376 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:35.699379 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:35.700470 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:35.700479 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:35.701651 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:35.701655 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:35.701756 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:35.701782 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 11:36:35.701838 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 11:36:35.702034 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"mutex_wait_us":7,"thread_start_us":103,"threads_started":1}
I20260806 11:36:35.702957 26414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:35.696822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59078 (local address 127.21.14.195:40495)
0806 11:36:35.697341 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:35.697352 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:35.697375 (+ 23us) server_negotiation.cc:407] Connection header received
0806 11:36:35.697450 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:35.697455 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:35.697672 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:35.697833 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:36:35.698095 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:35.699218 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:35.700633 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:35.701496 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:35.702435 (+ 939us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:35.702613 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:35.702666 (+ 53us) server_negotiation.cc:299] Negotiation successful
0806 11:36:35.702766 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":186,"threads_started":1}
I20260806 11:36:35.703626 26415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:35.697000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.193:35357 (local address 127.0.0.1:42586)
0806 11:36:35.697694 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:35.697707 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:35.697831 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:35.698429 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:35.698432 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:35.698450 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:36:35.698556 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:35.698563 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:35.700346 (+ 1783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:35.700349 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:35.701515 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:35.701529 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:35.703239 (+ 1710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:35.703242 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:35.703338 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:35.703361 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 11:36:35.703428 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 11:36:35.703499 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":632,"thread_start_us":89,"threads_started":1}
I20260806 11:36:35.704239 26416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:35.697326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42586 (local address 127.21.14.193:35357)
0806 11:36:35.697613 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:35.697621 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:35.698081 (+ 460us) server_negotiation.cc:407] Connection header received
0806 11:36:35.698151 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:35.698154 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:35.698297 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:35.698443 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:36:35.698650 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:35.700205 (+ 1555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:35.701669 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:35.702945 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:35.703059 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:35.703936 (+ 877us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:35.703992 (+ 56us) server_negotiation.cc:299] Negotiation successful
0806 11:36:35.704090 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":113,"threads_started":1}
I20260806 11:36:35.706112 26187 tablet_service.cc:1467] Processing CreateTablet for tablet 1a8f9416412348fd807c9068fb6a4a8f (DEFAULT_TABLE table=test-table [id=20e67ab987404003b4d98ae093d2b623]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:36:35.706215 26330 tablet_service.cc:1467] Processing CreateTablet for tablet 1a8f9416412348fd807c9068fb6a4a8f (DEFAULT_TABLE table=test-table [id=20e67ab987404003b4d98ae093d2b623]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:36:35.707340 26187 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a8f9416412348fd807c9068fb6a4a8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:35.707386 26330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a8f9416412348fd807c9068fb6a4a8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:35.707124 26048 tablet_service.cc:1467] Processing CreateTablet for tablet 1a8f9416412348fd807c9068fb6a4a8f (DEFAULT_TABLE table=test-table [id=20e67ab987404003b4d98ae093d2b623]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:36:35.708086 26048 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a8f9416412348fd807c9068fb6a4a8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:35.714540 26417 tablet_bootstrap.cc:492] T 1a8f9416412348fd807c9068fb6a4a8f P 7b34cb10ebfd4071b724a0d61188c817: Bootstrap starting.
I20260806 11:36:35.716020 26418 tablet_bootstrap.cc:492] T 1a8f9416412348fd807c9068fb6a4a8f P 8990cb0941cb43c7ae35da135e91ffc5: Bootstrap starting.
I20260806 11:36:35.716444 26419 tablet_bootstrap.cc:492] T 1a8f9416412348fd807c9068fb6a4a8f P da9c1362b3864546ac2207e9ed2606bb: Bootstrap starting.
I20260806 11:36:35.717063 26417 tablet_bootstrap.cc:654] T 1a8f9416412348fd807c9068fb6a4a8f P 7b34cb10ebfd4071b724a0d61188c817: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:35.718428 26417 log.cc:824] T 1a8f9416412348fd807c9068fb6a4a8f P 7b34cb10ebfd4071b724a0d61188c817: Log is configured to *not* fsync() on all Append() calls
I20260806 11:36:35.718784 26419 tablet_bootstrap.cc:654] T 1a8f9416412348fd807c9068fb6a4a8f P da9c1362b3864546ac2207e9ed2606bb: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:35.720057 26419 log.cc:824] T 1a8f9416412348fd807c9068fb6a4a8f P da9c1362b3864546ac2207e9ed2606bb: Log is configured to *not* fsync() on all Append() calls
I20260806 11:36:35.721108 26417 tablet_bootstrap.cc:492] T 1a8f9416412348fd807c9068fb6a4a8f P 7b34cb10ebfd4071b724a0d61188c817: No bootstrap required, opened a new log
I20260806 11:36:35.721347 26417 ts_tablet_manager.cc:1397] T 1a8f9416412348fd807c9068fb6a4a8f P 7b34cb10ebfd4071b724a0d61188c817: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 11:36:35.721376 26418 tablet_bootstrap.cc:654] T 1a8f9416412348fd807c9068fb6a4a8f P 8990cb0941cb43c7ae35da135e91ffc5: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:35.721985 26419 tablet_bootstrap.cc:492] T 1a8f9416412348fd807c9068fb6a4a8f P da9c1362b3864546ac2207e9ed2606bb: No bootstrap required, opened a new log
I20260806 11:36:35.722196 26419 ts_tablet_manager.cc:1397] T 1a8f9416412348fd807c9068fb6a4a8f P da9c1362b3864546ac2207e9ed2606bb: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 11:36:35.722242 26418 log.cc:824] T 1a8f9416412348fd807c9068fb6a4a8f P 8990cb0941cb43c7ae35da135e91ffc5: Log is configured to *not* fsync() on all Append() calls
I20260806 11:36:35.724428 26418 tablet_bootstrap.cc:492] T 1a8f9416412348fd807c9068fb6a4a8f P 8990cb0941cb43c7ae35da135e91ffc5: No bootstrap required, opened a new log
I20260806 11:36:35.724640 26418 ts_tablet_manager.cc:1397] T 1a8f9416412348fd807c9068fb6a4a8f P 8990cb0941cb43c7ae35da135e91ffc5: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20260806 11:36:35.725487 26417 raft_consensus.cc:348] T 1a8f9416412348fd807c9068fb6a4a8f P 7b34cb10ebfd4071b724a0d61188c817 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b34cb10ebfd4071b724a0d61188c817" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 45625 } } peers { permanent_uuid: "da9c1362b3864546ac2207e9ed2606bb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 40495 } } peers { permanent_uuid: "8990cb0941cb43c7ae35da135e91ffc5" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 35357 } }
I20260806 11:36:35.725540 26419 raft_consensus.cc:348] T 1a8f9416412348fd807c9068fb6a4a8f P da9c1362b3864546ac2207e9ed2606bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b34cb10ebfd4071b724a0d61188c817" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 45625 } } peers { permanent_uuid: "da9c1362b3864546ac2207e9ed2606bb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 40495 } } peers { permanent_uuid: "8990cb0941cb43c7ae35da135e91ffc5" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 35357 } }
I20260806 11:36:35.725809 26417 raft_consensus.cc:374] T 1a8f9416412348fd807c9068fb6a4a8f P 7b34cb10ebfd4071b724a0d61188c817 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:35.725860 26417 raft_consensus.cc:729] T 1a8f9416412348fd807c9068fb6a4a8f P 7b34cb10ebfd4071b724a0d61188c817 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b34cb10ebfd4071b724a0d61188c817, State: Initialized, Role: FOLLOWER
I20260806 11:36:35.725899 26419 raft_consensus.cc:374] T 1a8f9416412348fd807c9068fb6a4a8f P da9c1362b3864546ac2207e9ed2606bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:35.725973 26419 raft_consensus.cc:729] T 1a8f9416412348fd807c9068fb6a4a8f P da9c1362b3864546ac2207e9ed2606bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da9c1362b3864546ac2207e9ed2606bb, State: Initialized, Role: FOLLOWER
I20260806 11:36:35.726420 26419 consensus_queue.cc:260] T 1a8f9416412348fd807c9068fb6a4a8f P da9c1362b3864546ac2207e9ed2606bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b34cb10ebfd4071b724a0d61188c817" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 45625 } } peers { permanent_uuid: "da9c1362b3864546ac2207e9ed2606bb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 40495 } } peers { permanent_uuid: "8990cb0941cb43c7ae35da135e91ffc5" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 35357 } }
I20260806 11:36:35.726413 26417 consensus_queue.cc:260] T 1a8f9416412348fd807c9068fb6a4a8f P 7b34cb10ebfd4071b724a0d61188c817 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b34cb10ebfd4071b724a0d61188c817" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 45625 } } peers { permanent_uuid: "da9c1362b3864546ac2207e9ed2606bb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 40495 } } peers { permanent_uuid: "8990cb0941cb43c7ae35da135e91ffc5" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 35357 } }
I20260806 11:36:35.727197 26400 heartbeater.cc:503] Master 127.21.14.254:45651 was elected leader, sending a full tablet report...
I20260806 11:36:35.727540 26419 ts_tablet_manager.cc:1428] T 1a8f9416412348fd807c9068fb6a4a8f P da9c1362b3864546ac2207e9ed2606bb: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 11:36:35.727592 26417 ts_tablet_manager.cc:1428] T 1a8f9416412348fd807c9068fb6a4a8f P 7b34cb10ebfd4071b724a0d61188c817: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260806 11:36:35.727946 26255 heartbeater.cc:503] Master 127.21.14.254:45651 was elected leader, sending a full tablet report...
I20260806 11:36:35.729894 26418 raft_consensus.cc:348] T 1a8f9416412348fd807c9068fb6a4a8f P 8990cb0941cb43c7ae35da135e91ffc5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b34cb10ebfd4071b724a0d61188c817" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 45625 } } peers { permanent_uuid: "da9c1362b3864546ac2207e9ed2606bb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 40495 } } peers { permanent_uuid: "8990cb0941cb43c7ae35da135e91ffc5" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 35357 } }
I20260806 11:36:35.730188 26418 raft_consensus.cc:374] T 1a8f9416412348fd807c9068fb6a4a8f P 8990cb0941cb43c7ae35da135e91ffc5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:35.730249 26418 raft_consensus.cc:729] T 1a8f9416412348fd807c9068fb6a4a8f P 8990cb0941cb43c7ae35da135e91ffc5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8990cb0941cb43c7ae35da135e91ffc5, State: Initialized, Role: FOLLOWER
I20260806 11:36:35.730808 26418 consensus_queue.cc:260] T 1a8f9416412348fd807c9068fb6a4a8f P 8990cb0941cb43c7ae35da135e91ffc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b34cb10ebfd4071b724a0d61188c817" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 45625 } } peers { permanent_uuid: "da9c1362b3864546ac2207e9ed2606bb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 40495 } } peers { permanent_uuid: "8990cb0941cb43c7ae35da135e91ffc5" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 35357 } }
I20260806 11:36:35.731642 26117 heartbeater.cc:503] Master 127.21.14.254:45651 was elected leader, sending a full tablet report...
I20260806 11:36:35.731989 26418 ts_tablet_manager.cc:1428] T 1a8f9416412348fd807c9068fb6a4a8f P 8990cb0941cb43c7ae35da135e91ffc5: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
W20260806 11:36:35.887362 26118 tablet.cc:2369] T 1a8f9416412348fd807c9068fb6a4a8f P 8990cb0941cb43c7ae35da135e91ffc5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 11:36:35.891219 26256 tablet.cc:2369] T 1a8f9416412348fd807c9068fb6a4a8f P 7b34cb10ebfd4071b724a0d61188c817: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 11:36:35.893769 26401 tablet.cc:2369] T 1a8f9416412348fd807c9068fb6a4a8f P da9c1362b3864546ac2207e9ed2606bb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 11:36:35.978946 26425 raft_consensus.cc:482] T 1a8f9416412348fd807c9068fb6a4a8f P 8990cb0941cb43c7ae35da135e91ffc5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 11:36:35.979233 26425 raft_consensus.cc:504] T 1a8f9416412348fd807c9068fb6a4a8f P 8990cb0941cb43c7ae35da135e91ffc5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b34cb10ebfd4071b724a0d61188c817" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 45625 } } peers { permanent_uuid: "da9c1362b3864546ac2207e9ed2606bb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 40495 } } peers { permanent_uuid: "8990cb0941cb43c7ae35da135e91ffc5" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 35357 } }
I20260806 11:36:35.980955 26425 leader_election.cc:288] T 1a8f9416412348fd807c9068fb6a4a8f P 8990cb0941cb43c7ae35da135e91ffc5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7b34cb10ebfd4071b724a0d61188c817 (127.21.14.194:45625), da9c1362b3864546ac2207e9ed2606bb (127.21.14.195:40495)
I20260806 11:36:35.986358 26414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:35.981940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:43927 (local address 127.21.14.195:40495)
0806 11:36:35.982088 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:35.982099 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:35.982243 (+ 144us) server_negotiation.cc:407] Connection header received
0806 11:36:35.982388 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:35.982392 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:35.982456 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:35.982548 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:36:35.982995 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:35.983578 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:35.985223 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:35.985712 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:35.985861 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:35.985922 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:35.986128 (+ 206us) server_negotiation.cc:299] Negotiation successful
0806 11:36:35.986213 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 11:36:35.986392 26427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:35.981793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.195:40495 (local address 127.21.14.193:43927)
0806 11:36:35.982140 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:35.982154 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:35.982287 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:35.982561 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:35.982565 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:35.982584 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:36:35.982823 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:35.982834 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:35.983725 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:35.983732 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:35.985083 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:35.985094 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:35.985856 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:35.985860 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:35.985978 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:35.986003 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 11:36:35.986108 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 11:36:35.986192 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"mutex_wait_us":46,"spinlock_wait_cycles":22016,"thread_start_us":117,"threads_started":1}
I20260806 11:36:35.987491 26351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a8f9416412348fd807c9068fb6a4a8f" candidate_uuid: "8990cb0941cb43c7ae35da135e91ffc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da9c1362b3864546ac2207e9ed2606bb" is_pre_election: true
I20260806 11:36:35.987831 26412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:35.981423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:57491 (local address 127.21.14.194:45625)
0806 11:36:35.981594 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:35.981603 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:35.981881 (+ 278us) server_negotiation.cc:407] Connection header received
0806 11:36:35.982287 (+ 406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:35.982292 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:35.982351 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:35.982469 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:36:35.983054 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:35.983952 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:35.985760 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:35.986561 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:35.986681 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:35.986752 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:35.987551 (+ 799us) server_negotiation.cc:299] Negotiation successful
0806 11:36:35.987666 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 11:36:35.987905 26351 raft_consensus.cc:2454] T 1a8f9416412348fd807c9068fb6a4a8f P da9c1362b3864546ac2207e9ed2606bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8990cb0941cb43c7ae35da135e91ffc5 in term 0.
I20260806 11:36:35.988404 26426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:35.981305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.194:45625 (local address 127.21.14.193:57491)
0806 11:36:35.981781 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:35.981797 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:35.982129 (+ 332us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:35.982618 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:35.982623 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:35.982674 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:36:35.982891 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:35.982902 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:35.984130 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:35.984154 (+ 24us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:35.985584 (+ 1430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:35.985600 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:35.986912 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:35.986919 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:35.987347 (+ 428us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:35.987388 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 11:36:35.988073 (+ 685us) client_negotiation.cc:240] Negotiation successful
0806 11:36:35.988189 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":398,"mutex_wait_us":5,"spinlock_wait_cycles":896,"thread_start_us":131,"threads_started":1}
I20260806 11:36:35.988595 25998 leader_election.cc:302] T 1a8f9416412348fd807c9068fb6a4a8f P 8990cb0941cb43c7ae35da135e91ffc5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8990cb0941cb43c7ae35da135e91ffc5, da9c1362b3864546ac2207e9ed2606bb; no voters:
I20260806 11:36:35.988955 26425 raft_consensus.cc:2781] T 1a8f9416412348fd807c9068fb6a4a8f P 8990cb0941cb43c7ae35da135e91ffc5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 11:36:35.989061 26425 raft_consensus.cc:482] T 1a8f9416412348fd807c9068fb6a4a8f P 8990cb0941cb43c7ae35da135e91ffc5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 11:36:35.989120 26425 raft_consensus.cc:3037] T 1a8f9416412348fd807c9068fb6a4a8f P 8990cb0941cb43c7ae35da135e91ffc5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:36:35.989475 26208 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a8f9416412348fd807c9068fb6a4a8f" candidate_uuid: "8990cb0941cb43c7ae35da135e91ffc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b34cb10ebfd4071b724a0d61188c817" is_pre_election: true
I20260806 11:36:35.989789 26208 raft_consensus.cc:2454] T 1a8f9416412348fd807c9068fb6a4a8f P 7b34cb10ebfd4071b724a0d61188c817 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8990cb0941cb43c7ae35da135e91ffc5 in term 0.
I20260806 11:36:35.990657 26425 raft_consensus.cc:504] T 1a8f9416412348fd807c9068fb6a4a8f P 8990cb0941cb43c7ae35da135e91ffc5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b34cb10ebfd4071b724a0d61188c817" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 45625 } } peers { permanent_uuid: "da9c1362b3864546ac2207e9ed2606bb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 40495 } } peers { permanent_uuid: "8990cb0941cb43c7ae35da135e91ffc5" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 35357 } }
I20260806 11:36:35.991166 26425 leader_election.cc:288] T 1a8f9416412348fd807c9068fb6a4a8f P 8990cb0941cb43c7ae35da135e91ffc5 [CANDIDATE]: Term 1 election: Requested vote from peers 7b34cb10ebfd4071b724a0d61188c817 (127.21.14.194:45625), da9c1362b3864546ac2207e9ed2606bb (127.21.14.195:40495)
I20260806 11:36:35.991659 26208 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a8f9416412348fd807c9068fb6a4a8f" candidate_uuid: "8990cb0941cb43c7ae35da135e91ffc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b34cb10ebfd4071b724a0d61188c817"
I20260806 11:36:35.991714 26351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a8f9416412348fd807c9068fb6a4a8f" candidate_uuid: "8990cb0941cb43c7ae35da135e91ffc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da9c1362b3864546ac2207e9ed2606bb"
I20260806 11:36:35.991766 26208 raft_consensus.cc:3037] T 1a8f9416412348fd807c9068fb6a4a8f P 7b34cb10ebfd4071b724a0d61188c817 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:36:35.991894 26351 raft_consensus.cc:3037] T 1a8f9416412348fd807c9068fb6a4a8f P da9c1362b3864546ac2207e9ed2606bb [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:36:35.992900 26208 raft_consensus.cc:2454] T 1a8f9416412348fd807c9068fb6a4a8f P 7b34cb10ebfd4071b724a0d61188c817 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8990cb0941cb43c7ae35da135e91ffc5 in term 1.
I20260806 11:36:35.993201 26351 raft_consensus.cc:2454] T 1a8f9416412348fd807c9068fb6a4a8f P da9c1362b3864546ac2207e9ed2606bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8990cb0941cb43c7ae35da135e91ffc5 in term 1.
I20260806 11:36:35.993261 25998 leader_election.cc:302] T 1a8f9416412348fd807c9068fb6a4a8f P 8990cb0941cb43c7ae35da135e91ffc5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7b34cb10ebfd4071b724a0d61188c817, 8990cb0941cb43c7ae35da135e91ffc5; no voters:
I20260806 11:36:35.993635 26425 raft_consensus.cc:2781] T 1a8f9416412348fd807c9068fb6a4a8f P 8990cb0941cb43c7ae35da135e91ffc5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:36:35.993898 26425 raft_consensus.cc:686] T 1a8f9416412348fd807c9068fb6a4a8f P 8990cb0941cb43c7ae35da135e91ffc5 [term 1 LEADER]: Becoming Leader. State: Replica: 8990cb0941cb43c7ae35da135e91ffc5, State: Running, Role: LEADER
I20260806 11:36:35.994666 26425 consensus_queue.cc:237] T 1a8f9416412348fd807c9068fb6a4a8f P 8990cb0941cb43c7ae35da135e91ffc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b34cb10ebfd4071b724a0d61188c817" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 45625 } } peers { permanent_uuid: "da9c1362b3864546ac2207e9ed2606bb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 40495 } } peers { permanent_uuid: "8990cb0941cb43c7ae35da135e91ffc5" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 35357 } }
I20260806 11:36:35.999246 25918 catalog_manager.cc:5495] T 1a8f9416412348fd807c9068fb6a4a8f P 8990cb0941cb43c7ae35da135e91ffc5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8990cb0941cb43c7ae35da135e91ffc5 (127.21.14.193). New cstate: current_term: 1 leader_uuid: "8990cb0941cb43c7ae35da135e91ffc5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b34cb10ebfd4071b724a0d61188c817" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 45625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da9c1362b3864546ac2207e9ed2606bb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 40495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8990cb0941cb43c7ae35da135e91ffc5" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 35357 } health_report { overall_health: HEALTHY } } }
I20260806 11:36:36.069558 26416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:36.065617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42600 (local address 127.21.14.193:35357)
0806 11:36:36.065762 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:36.065767 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:36.065782 (+ 15us) server_negotiation.cc:407] Connection header received
0806 11:36:36.065908 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:36.065910 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:36.065969 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:36.066098 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 11:36:36.066367 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:36.067011 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:36.067974 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:36.068202 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:36.068319 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:36.068670 (+ 351us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 11:36:36.069131 (+ 461us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 11:36:36.069276 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:36.069339 (+ 63us) server_negotiation.cc:299] Negotiation successful
0806 11:36:36.069401 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 11:36:36.076414 26351 raft_consensus.cc:1261] T 1a8f9416412348fd807c9068fb6a4a8f P da9c1362b3864546ac2207e9ed2606bb [term 1 FOLLOWER]: Refusing update from remote peer 8990cb0941cb43c7ae35da135e91ffc5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 11:36:36.076416 26208 raft_consensus.cc:1261] T 1a8f9416412348fd807c9068fb6a4a8f P 7b34cb10ebfd4071b724a0d61188c817 [term 1 FOLLOWER]: Refusing update from remote peer 8990cb0941cb43c7ae35da135e91ffc5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 11:36:36.077306 26428 consensus_queue.cc:1035] T 1a8f9416412348fd807c9068fb6a4a8f P 8990cb0941cb43c7ae35da135e91ffc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da9c1362b3864546ac2207e9ed2606bb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 40495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 11:36:36.077564 26425 consensus_queue.cc:1035] T 1a8f9416412348fd807c9068fb6a4a8f P 8990cb0941cb43c7ae35da135e91ffc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b34cb10ebfd4071b724a0d61188c817" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 45625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 11:36:36.085642 26433 mvcc.cc:205] Tried to move back new op lower bound from 7315522339119951872 to 7315522338798190592. Current Snapshot: MvccSnapshot[applied={T|T < 7315522339119951872}]
I20260806 11:36:36.095547 25910 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53932:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 11:36:36.095786 25910 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53932:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 11:36:36.099483 25910 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1ada5e0a765d49a38476b82c8653405d: Sending DeleteTablet for 3 replicas of tablet 1a8f9416412348fd807c9068fb6a4a8f
I20260806 11:36:36.100580 26187 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a8f9416412348fd807c9068fb6a4a8f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:36:36 UTC) from {username='kudu', principal='kudu/127.21.14.254@KRBTEST.COM'} at 127.0.0.1:45408
I20260806 11:36:36.100761 26330 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a8f9416412348fd807c9068fb6a4a8f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:36:36 UTC) from {username='kudu', principal='kudu/127.21.14.254@KRBTEST.COM'} at 127.0.0.1:59078
I20260806 11:36:36.101361 26442 tablet_replica.cc:330] T 1a8f9416412348fd807c9068fb6a4a8f P da9c1362b3864546ac2207e9ed2606bb: stopping tablet replica
I20260806 11:36:36.101478 26440 tablet_replica.cc:330] T 1a8f9416412348fd807c9068fb6a4a8f P 7b34cb10ebfd4071b724a0d61188c817: stopping tablet replica
I20260806 11:36:36.101540 26048 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a8f9416412348fd807c9068fb6a4a8f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:36:36 UTC) from {username='kudu', principal='kudu/127.21.14.254@KRBTEST.COM'} at 127.0.0.1:42586
I20260806 11:36:36.101866 26442 raft_consensus.cc:2229] T 1a8f9416412348fd807c9068fb6a4a8f P da9c1362b3864546ac2207e9ed2606bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 11:36:36.101866 26440 raft_consensus.cc:2229] T 1a8f9416412348fd807c9068fb6a4a8f P 7b34cb10ebfd4071b724a0d61188c817 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 11:36:36.102166 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 25982
I20260806 11:36:36.102269 26440 raft_consensus.cc:2258] T 1a8f9416412348fd807c9068fb6a4a8f P 7b34cb10ebfd4071b724a0d61188c817 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 11:36:36.104185 26442 raft_consensus.cc:2258] T 1a8f9416412348fd807c9068fb6a4a8f P da9c1362b3864546ac2207e9ed2606bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 11:36:36.104926 26440 ts_tablet_manager.cc:1905] T 1a8f9416412348fd807c9068fb6a4a8f P 7b34cb10ebfd4071b724a0d61188c817: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 11:36:36.106331 26442 ts_tablet_manager.cc:1905] T 1a8f9416412348fd807c9068fb6a4a8f P da9c1362b3864546ac2207e9ed2606bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 11:36:36.108824 26440 ts_tablet_manager.cc:1918] T 1a8f9416412348fd807c9068fb6a4a8f P 7b34cb10ebfd4071b724a0d61188c817: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 11:36:36.108968 26440 log.cc:1196] T 1a8f9416412348fd807c9068fb6a4a8f P 7b34cb10ebfd4071b724a0d61188c817: Deleting WAL directory at /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/minicluster-data/ts-1/wal/wals/1a8f9416412348fd807c9068fb6a4a8f
I20260806 11:36:36.109395 26440 ts_tablet_manager.cc:1939] T 1a8f9416412348fd807c9068fb6a4a8f P 7b34cb10ebfd4071b724a0d61188c817: Deleting consensus metadata
I20260806 11:36:36.109938 26442 ts_tablet_manager.cc:1918] T 1a8f9416412348fd807c9068fb6a4a8f P da9c1362b3864546ac2207e9ed2606bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 11:36:36.110114 26442 log.cc:1196] T 1a8f9416412348fd807c9068fb6a4a8f P da9c1362b3864546ac2207e9ed2606bb: Deleting WAL directory at /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786016169045816-21563-0/minicluster-data/ts-2/wal/wals/1a8f9416412348fd807c9068fb6a4a8f
I20260806 11:36:36.110512 26442 ts_tablet_manager.cc:1939] T 1a8f9416412348fd807c9068fb6a4a8f P da9c1362b3864546ac2207e9ed2606bb: Deleting consensus metadata
I20260806 11:36:36.111421 25902 catalog_manager.cc:4841] TS 7b34cb10ebfd4071b724a0d61188c817 (127.21.14.194:45625): tablet 1a8f9416412348fd807c9068fb6a4a8f (table test-table [id=20e67ab987404003b4d98ae093d2b623]) successfully deleted
I20260806 11:36:36.112087 25902 catalog_manager.cc:4841] TS da9c1362b3864546ac2207e9ed2606bb (127.21.14.195:40495): tablet 1a8f9416412348fd807c9068fb6a4a8f (table test-table [id=20e67ab987404003b4d98ae093d2b623]) successfully deleted
W20260806 11:36:36.116338 25902 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.14.193:35357 (error 108)
I20260806 11:36:36.116494 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 26122
I20260806 11:36:36.119335 26415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:36.118897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.193:35357 (local address 127.0.0.1:42612)
0806 11:36:36.119075 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:36.119194 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.193:35357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260806 11:36:36.119745 25902 catalog_manager.cc:4584] TS 8990cb0941cb43c7ae35da135e91ffc5 (127.21.14.193:35357): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1a8f9416412348fd807c9068fb6a4a8f: Network error: Client connection negotiation failed: client connection to 127.21.14.193:35357: connect: Connection refused (error 111)
I20260806 11:36:36.125386 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 26260
I20260806 11:36:36.134323 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 25886
2026-08-06T11:36:36Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2810 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-06T11:36:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T11:36:36Z Disabled control of system clock
I20260806 11:36:36.172845 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786016169045816-21563-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786016169045816-21563-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786016169045816-21563-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:33271
--webserver_interface=127.21.14.254
--webserver_port=0
--builtin_ntp_servers=127.21.14.212:44921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.14.254:33271
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 11:36:36.303154 26449 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:36:36.303781 26449 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:36:36.303838 26449 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:36:36.308240 26449 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 11:36:36.308323 26449 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:36:36.308341 26449 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:36:36.308357 26449 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 11:36:36.308373 26449 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 11:36:36.314200 26449 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:44921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786016169045816-21563-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786016169045816-21563-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.14.254:33271
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:33271
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.14.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26449
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786016169045816-21563-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:36:36.315685 26449 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:36:36.318008 26449 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:36:36.327661 26455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:36.327725 26449 server_base.cc:1034] running on GCE node
W20260806 11:36:36.327656 26457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:36.327672 26454 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:36.328593 26449 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:36:36.329802 26449 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:36:36.330968 26449 hybrid_clock.cc:648] HybridClock initialized: now 1786016196330954 us; error 28 us; skew 500 ppm
I20260806 11:36:36.333079 26449 webserver.cc:460] Webserver started at http://127.21.14.254:39543/ using document root <none> and password file <none>
I20260806 11:36:36.333683 26449 fs_manager.cc:359] Metadata directory not provided
I20260806 11:36:36.333747 26449 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:36:36.333937 26449 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:36:36.336215 26449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786016169045816-21563-0/minicluster-data/master-0/data/instance:
uuid: "8d6466c9e62046d1a3d5ca9b20f8b569"
format_stamp: "Formatted at 2026-08-06 11:36:36 on dist-test-slave-7c35"
server_key: "0aa8ae4cf0820349c7204869875a4fae"
server_key_iv: "d6a2d6d37495660d47ae98e1b8761e64"
server_key_version: "encryptionkey@0"
I20260806 11:36:36.336773 26449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786016169045816-21563-0/minicluster-data/master-0/wal/instance:
uuid: "8d6466c9e62046d1a3d5ca9b20f8b569"
format_stamp: "Formatted at 2026-08-06 11:36:36 on dist-test-slave-7c35"
server_key: "0aa8ae4cf0820349c7204869875a4fae"
server_key_iv: "d6a2d6d37495660d47ae98e1b8761e64"
server_key_version: "encryptionkey@0"
I20260806 11:36:36.340849 26449 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 11:36:36.343780 26463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:36.345404 26449 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 11:36:36.345570 26449 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786016169045816-21563-0/minicluster-data/master-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786016169045816-21563-0/minicluster-data/master-0/wal
uuid: "8d6466c9e62046d1a3d5ca9b20f8b569"
format_stamp: "Formatted at 2026-08-06 11:36:36 on dist-test-slave-7c35"
server_key: "0aa8ae4cf0820349c7204869875a4fae"
server_key_iv: "d6a2d6d37495660d47ae98e1b8761e64"
server_key_version: "encryptionkey@0"
I20260806 11:36:36.345687 26449 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786016169045816-21563-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786016169045816-21563-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786016169045816-21563-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:36:36.350844 26449 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:36:36.351544 26449 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:36:36.351701 26449 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:36:36.361280 26449 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.254:33271
I20260806 11:36:36.361332 26516 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.254:33271 every 8 connection(s)
I20260806 11:36:36.362531 26449 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
I20260806 11:36:36.366968 26517 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:36.371426 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 26449
I20260806 11:36:36.371588 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786016169045816-21563-0/minicluster-data/master-0/wal/instance
I20260806 11:36:36.371905 21563 external_mini_cluster.cc:1428] Setting key 20828466daa82963ed0a6243ad706584
I20260806 11:36:36.374603 26517 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d6466c9e62046d1a3d5ca9b20f8b569: Bootstrap starting.
I20260806 11:36:36.377591 26517 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8d6466c9e62046d1a3d5ca9b20f8b569: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:36.378463 26517 log.cc:824] T 00000000000000000000000000000000 P 8d6466c9e62046d1a3d5ca9b20f8b569: Log is configured to *not* fsync() on all Append() calls
I20260806 11:36:36.378540 26520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:36.373698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51290 (local address 127.21.14.254:33271)
0806 11:36:36.374446 (+ 748us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:36.374464 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:36.374503 (+ 39us) server_negotiation.cc:407] Connection header received
0806 11:36:36.375343 (+ 840us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:36.375367 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:36.375781 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:36.376104 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:36.377061 (+ 957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:36.377066 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:36.377079 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 11:36:36.377089 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:36.377149 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 11:36:36.377166 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:36.377257 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:36.377447 (+ 190us) server_negotiation.cc:299] Negotiation successful
0806 11:36:36.377755 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":180,"threads_started":1}
I20260806 11:36:36.380928 26517 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d6466c9e62046d1a3d5ca9b20f8b569: No bootstrap required, opened a new log
I20260806 11:36:36.385221 26517 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8d6466c9e62046d1a3d5ca9b20f8b569 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d6466c9e62046d1a3d5ca9b20f8b569" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 33271 } }
I20260806 11:36:36.385490 26517 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8d6466c9e62046d1a3d5ca9b20f8b569 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:36.385537 26517 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8d6466c9e62046d1a3d5ca9b20f8b569 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d6466c9e62046d1a3d5ca9b20f8b569, State: Initialized, Role: FOLLOWER
I20260806 11:36:36.385936 26517 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8d6466c9e62046d1a3d5ca9b20f8b569 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d6466c9e62046d1a3d5ca9b20f8b569" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 33271 } }
I20260806 11:36:36.386056 26517 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8d6466c9e62046d1a3d5ca9b20f8b569 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 11:36:36.386098 26517 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8d6466c9e62046d1a3d5ca9b20f8b569 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 11:36:36.386174 26517 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8d6466c9e62046d1a3d5ca9b20f8b569 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:36:36.387286 26517 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8d6466c9e62046d1a3d5ca9b20f8b569 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d6466c9e62046d1a3d5ca9b20f8b569" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 33271 } }
I20260806 11:36:36.387686 26517 leader_election.cc:302] T 00000000000000000000000000000000 P 8d6466c9e62046d1a3d5ca9b20f8b569 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8d6466c9e62046d1a3d5ca9b20f8b569; no voters:
I20260806 11:36:36.388161 26517 leader_election.cc:288] T 00000000000000000000000000000000 P 8d6466c9e62046d1a3d5ca9b20f8b569 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 11:36:36.388322 26522 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8d6466c9e62046d1a3d5ca9b20f8b569 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:36:36.388625 26522 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8d6466c9e62046d1a3d5ca9b20f8b569 [term 1 LEADER]: Becoming Leader. State: Replica: 8d6466c9e62046d1a3d5ca9b20f8b569, State: Running, Role: LEADER
I20260806 11:36:36.388948 26522 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8d6466c9e62046d1a3d5ca9b20f8b569 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d6466c9e62046d1a3d5ca9b20f8b569" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 33271 } }
I20260806 11:36:36.389580 26517 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8d6466c9e62046d1a3d5ca9b20f8b569 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 11:36:36.390795 26523 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d6466c9e62046d1a3d5ca9b20f8b569 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8d6466c9e62046d1a3d5ca9b20f8b569" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d6466c9e62046d1a3d5ca9b20f8b569" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 33271 } } }
I20260806 11:36:36.390956 26523 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d6466c9e62046d1a3d5ca9b20f8b569 [sys.catalog]: This master's current role is: LEADER
I20260806 11:36:36.391181 26524 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d6466c9e62046d1a3d5ca9b20f8b569 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8d6466c9e62046d1a3d5ca9b20f8b569. Latest consensus state: current_term: 1 leader_uuid: "8d6466c9e62046d1a3d5ca9b20f8b569" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d6466c9e62046d1a3d5ca9b20f8b569" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 33271 } } }
I20260806 11:36:36.391357 26524 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d6466c9e62046d1a3d5ca9b20f8b569 [sys.catalog]: This master's current role is: LEADER
I20260806 11:36:36.391424 26531 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 11:36:36.394986 26531 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 11:36:36.401482 26531 catalog_manager.cc:1347] Generated new cluster ID: b95dbddbea6d4020a368ce80b816d057
I20260806 11:36:36.401659 26531 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 11:36:36.425580 26531 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 11:36:36.425969 26531 catalog_manager.cc:1504] Loading token signing keys...
I20260806 11:36:36.433935 26531 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8d6466c9e62046d1a3d5ca9b20f8b569: Generated new TSK 0
I20260806 11:36:36.435050 26531 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 11:36:36.449498 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786016169045816-21563-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786016169045816-21563-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786016169045816-21563-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.193:0
--local_ip_for_outbound_sockets=127.21.14.193
--webserver_interface=127.21.14.193
--webserver_port=0
--tserver_master_addrs=127.21.14.254:33271
--builtin_ntp_servers=127.21.14.212:44921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 11:36:36.577855 26541 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:36:36.578164 26541 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:36:36.578207 26541 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:36:36.582409 26541 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:36:36.582500 26541 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:36:36.582562 26541 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.193
I20260806 11:36:36.587720 26541 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:44921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786016169045816-21563-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786016169045816-21563-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.14.193
--webserver_port=0
--tserver_master_addrs=127.21.14.254:33271
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26541
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.193
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786016169045816-21563-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:36:36.588949 26541 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:36:36.590602 26541 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:36:36.602522 26547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:36.602545 26548 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:36.602586 26550 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:36.603051 26541 server_base.cc:1034] running on GCE node
I20260806 11:36:36.604453 26541 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:36:36.605297 26541 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:36:36.606534 26541 hybrid_clock.cc:648] HybridClock initialized: now 1786016196606480 us; error 70 us; skew 500 ppm
I20260806 11:36:36.608819 26541 webserver.cc:460] Webserver started at http://127.21.14.193:43419/ using document root <none> and password file <none>
I20260806 11:36:36.609498 26541 fs_manager.cc:359] Metadata directory not provided
I20260806 11:36:36.609611 26541 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:36:36.609877 26541 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:36:36.612689 26541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786016169045816-21563-0/minicluster-data/ts-0/data/instance:
uuid: "be2c4aef62d7499d85c9a4f6eff03008"
format_stamp: "Formatted at 2026-08-06 11:36:36 on dist-test-slave-7c35"
server_key: "ec3405e91889e562c16dbf77ab584198"
server_key_iv: "72dc0ed68d1467e80d2d0ac2b79bc152"
server_key_version: "encryptionkey@0"
I20260806 11:36:36.613462 26541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786016169045816-21563-0/minicluster-data/ts-0/wal/instance:
uuid: "be2c4aef62d7499d85c9a4f6eff03008"
format_stamp: "Formatted at 2026-08-06 11:36:36 on dist-test-slave-7c35"
server_key: "ec3405e91889e562c16dbf77ab584198"
server_key_iv: "72dc0ed68d1467e80d2d0ac2b79bc152"
server_key_version: "encryptionkey@0"
I20260806 11:36:36.618188 26541 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 11:36:36.621634 26556 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:36.623660 26541 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 11:36:36.623857 26541 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786016169045816-21563-0/minicluster-data/ts-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786016169045816-21563-0/minicluster-data/ts-0/wal
uuid: "be2c4aef62d7499d85c9a4f6eff03008"
format_stamp: "Formatted at 2026-08-06 11:36:36 on dist-test-slave-7c35"
server_key: "ec3405e91889e562c16dbf77ab584198"
server_key_iv: "72dc0ed68d1467e80d2d0ac2b79bc152"
server_key_version: "encryptionkey@0"
I20260806 11:36:36.624028 26541 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786016169045816-21563-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786016169045816-21563-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786016169045816-21563-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:36:36.629595 26541 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:36:36.630429 26541 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:36:36.630667 26541 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:36:36.631439 26541 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:36:36.632539 26541 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:36:36.632615 26541 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:36.632692 26541 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:36:36.632735 26541 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:36.643306 26541 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.193:46069
I20260806 11:36:36.643323 26675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.193:46069 every 8 connection(s)
I20260806 11:36:36.644377 26541 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
I20260806 11:36:36.648021 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 26541
I20260806 11:36:36.648178 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786016169045816-21563-0/minicluster-data/ts-0/wal/instance
I20260806 11:36:36.648496 21563 external_mini_cluster.cc:1428] Setting key c61e2fc332a3cf48eb47955d81726bb2
I20260806 11:36:36.651234 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786016169045816-21563-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786016169045816-21563-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786016169045816-21563-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.194:0
--local_ip_for_outbound_sockets=127.21.14.194
--webserver_interface=127.21.14.194
--webserver_port=0
--tserver_master_addrs=127.21.14.254:33271
--builtin_ntp_servers=127.21.14.212:44921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260806 11:36:36.651929 26520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:36.646531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:38841 (local address 127.21.14.254:33271)
0806 11:36:36.646690 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:36.646694 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:36.648156 (+ 1462us) server_negotiation.cc:407] Connection header received
0806 11:36:36.649893 (+ 1737us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:36.649896 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:36.649941 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:36.650035 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:36.651395 (+ 1360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:36.651399 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:36.651402 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 11:36:36.651415 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:36.651443 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 11:36:36.651458 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:36.651562 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:36.651740 (+ 178us) server_negotiation.cc:299] Negotiation successful
0806 11:36:36.651791 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 11:36:36.652945 26680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:36.647097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:33271 (local address 127.21.14.193:38841)
0806 11:36:36.647975 (+ 878us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:36.648038 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:36.649664 (+ 1626us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:36.650243 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:36.650252 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:36.650359 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:36.651095 (+ 736us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 11:36:36.651097 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:36.651147 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 11:36:36.651232 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:36.651575 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:36.651589 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:36.651600 (+ 11us) client_negotiation.cc:769] Sending connection context
0806 11:36:36.651797 (+ 197us) client_negotiation.cc:240] Negotiation successful
0806 11:36:36.652093 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":157,"threads_started":1}
I20260806 11:36:36.654362 26676 heartbeater.cc:347] Connected to a master server at 127.21.14.254:33271
I20260806 11:36:36.654666 26676 heartbeater.cc:464] Registering TS with master...
I20260806 11:36:36.655200 26676 heartbeater.cc:511] Master 127.21.14.254:33271 requested a full tablet report, sending...
I20260806 11:36:36.656953 26481 ts_manager.cc:194] Registered new tserver with Master: be2c4aef62d7499d85c9a4f6eff03008 (127.21.14.193:46069)
W20260806 11:36:36.773120 26681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:36:36.773476 26681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:36:36.773571 26681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:36:36.777827 26681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:36:36.777937 26681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:36:36.778038 26681 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.194
I20260806 11:36:36.783449 26681 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:44921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786016169045816-21563-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786016169045816-21563-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.14.194
--webserver_port=0
--tserver_master_addrs=127.21.14.254:33271
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26681
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.194
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786016169045816-21563-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:36:36.784688 26681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:36:36.786212 26681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:36:36.797017 26687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:36.797017 26686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:36.797020 26689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:36.797483 26681 server_base.cc:1034] running on GCE node
I20260806 11:36:36.797931 26681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:36:36.798672 26681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:36:36.799876 26681 hybrid_clock.cc:648] HybridClock initialized: now 1786016196799844 us; error 44 us; skew 500 ppm
I20260806 11:36:36.802006 26681 webserver.cc:460] Webserver started at http://127.21.14.194:34825/ using document root <none> and password file <none>
I20260806 11:36:36.802785 26681 fs_manager.cc:359] Metadata directory not provided
I20260806 11:36:36.802953 26681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:36:36.803236 26681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:36:36.805606 26681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786016169045816-21563-0/minicluster-data/ts-1/data/instance:
uuid: "91c66f260f724672839f57beb0819074"
format_stamp: "Formatted at 2026-08-06 11:36:36 on dist-test-slave-7c35"
server_key: "ff3311f9b11f7108daedd73d6779eadb"
server_key_iv: "d8f506f1cda895b71074f6810ae55794"
server_key_version: "encryptionkey@0"
I20260806 11:36:36.806303 26681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786016169045816-21563-0/minicluster-data/ts-1/wal/instance:
uuid: "91c66f260f724672839f57beb0819074"
format_stamp: "Formatted at 2026-08-06 11:36:36 on dist-test-slave-7c35"
server_key: "ff3311f9b11f7108daedd73d6779eadb"
server_key_iv: "d8f506f1cda895b71074f6810ae55794"
server_key_version: "encryptionkey@0"
I20260806 11:36:36.810968 26681 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260806 11:36:36.814145 26695 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:36.815814 26681 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260806 11:36:36.816025 26681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786016169045816-21563-0/minicluster-data/ts-1/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786016169045816-21563-0/minicluster-data/ts-1/wal
uuid: "91c66f260f724672839f57beb0819074"
format_stamp: "Formatted at 2026-08-06 11:36:36 on dist-test-slave-7c35"
server_key: "ff3311f9b11f7108daedd73d6779eadb"
server_key_iv: "d8f506f1cda895b71074f6810ae55794"
server_key_version: "encryptionkey@0"
I20260806 11:36:36.816188 26681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786016169045816-21563-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786016169045816-21563-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786016169045816-21563-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:36:36.822072 26681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:36:36.822829 26681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:36:36.823249 26681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:36:36.823951 26681 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:36:36.825007 26681 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:36:36.825084 26681 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:36.825158 26681 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:36:36.825212 26681 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:36.835064 26681 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.194:39973
I20260806 11:36:36.835109 26813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.194:39973 every 8 connection(s)
I20260806 11:36:36.836432 26681 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
I20260806 11:36:36.840229 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 26681
I20260806 11:36:36.840394 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786016169045816-21563-0/minicluster-data/ts-1/wal/instance
I20260806 11:36:36.840788 21563 external_mini_cluster.cc:1428] Setting key d5193bd39b355b22f0c7fd174d53c0f1
I20260806 11:36:36.844221 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786016169045816-21563-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786016169045816-21563-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786016169045816-21563-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.195:0
--local_ip_for_outbound_sockets=127.21.14.195
--webserver_interface=127.21.14.195
--webserver_port=0
--tserver_master_addrs=127.21.14.254:33271
--builtin_ntp_servers=127.21.14.212:44921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260806 11:36:36.846812 26520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:36.838751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:43801 (local address 127.21.14.254:33271)
0806 11:36:36.838964 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:36.838970 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:36.840811 (+ 1841us) server_negotiation.cc:407] Connection header received
0806 11:36:36.843362 (+ 2551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:36.843365 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:36.843409 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:36.843478 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:36.845927 (+ 2449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:36.845931 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:36.845934 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 11:36:36.845950 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:36.845977 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 11:36:36.845987 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:36.846085 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:36.846483 (+ 398us) server_negotiation.cc:299] Negotiation successful
0806 11:36:36.846541 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 11:36:36.847991 26816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:36.839375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:33271 (local address 127.21.14.194:43801)
0806 11:36:36.840557 (+ 1182us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:36.840635 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:36.842944 (+ 2309us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:36.843746 (+ 802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:36.843765 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:36.844131 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:36.845229 (+ 1098us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 11:36:36.845236 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:36.845499 (+ 263us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 11:36:36.845659 (+ 160us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:36.846104 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:36.846117 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:36.846129 (+ 12us) client_negotiation.cc:769] Sending connection context
0806 11:36:36.846494 (+ 365us) client_negotiation.cc:240] Negotiation successful
0806 11:36:36.846928 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":161,"threads_started":1}
I20260806 11:36:36.849954 26814 heartbeater.cc:347] Connected to a master server at 127.21.14.254:33271
I20260806 11:36:36.850334 26814 heartbeater.cc:464] Registering TS with master...
I20260806 11:36:36.850986 26814 heartbeater.cc:511] Master 127.21.14.254:33271 requested a full tablet report, sending...
I20260806 11:36:36.852571 26481 ts_manager.cc:194] Registered new tserver with Master: 91c66f260f724672839f57beb0819074 (127.21.14.194:39973)
W20260806 11:36:36.971738 26817 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:36:36.972052 26817 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:36:36.972095 26817 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:36:36.976187 26817 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:36:36.976248 26817 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:36:36.976307 26817 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.195
I20260806 11:36:36.981410 26817 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:44921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786016169045816-21563-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786016169045816-21563-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.14.195
--webserver_port=0
--tserver_master_addrs=127.21.14.254:33271
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26817
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.195
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786016169045816-21563-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:36:36.982599 26817 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:36:36.984120 26817 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:36:36.995409 26823 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:36.995429 26825 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:36.995424 26822 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:36.995616 26817 server_base.cc:1034] running on GCE node
I20260806 11:36:36.996289 26817 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:36:36.997107 26817 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:36:36.998337 26817 hybrid_clock.cc:648] HybridClock initialized: now 1786016196998321 us; error 44 us; skew 500 ppm
I20260806 11:36:37.000592 26817 webserver.cc:460] Webserver started at http://127.21.14.195:40551/ using document root <none> and password file <none>
I20260806 11:36:37.001296 26817 fs_manager.cc:359] Metadata directory not provided
I20260806 11:36:37.001405 26817 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:36:37.001667 26817 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:36:37.004114 26817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786016169045816-21563-0/minicluster-data/ts-2/data/instance:
uuid: "65b7ce1ba0a44b25a58db9e94b90014b"
format_stamp: "Formatted at 2026-08-06 11:36:36 on dist-test-slave-7c35"
server_key: "d82b1c0ededfa2bf8aff3c72104fae12"
server_key_iv: "b650671a18ee3a237f1ed35c54d39624"
server_key_version: "encryptionkey@0"
I20260806 11:36:37.004762 26817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786016169045816-21563-0/minicluster-data/ts-2/wal/instance:
uuid: "65b7ce1ba0a44b25a58db9e94b90014b"
format_stamp: "Formatted at 2026-08-06 11:36:36 on dist-test-slave-7c35"
server_key: "d82b1c0ededfa2bf8aff3c72104fae12"
server_key_iv: "b650671a18ee3a237f1ed35c54d39624"
server_key_version: "encryptionkey@0"
I20260806 11:36:37.009153 26817 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 11:36:37.012430 26831 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:37.013883 26817 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 11:36:37.014066 26817 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786016169045816-21563-0/minicluster-data/ts-2/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786016169045816-21563-0/minicluster-data/ts-2/wal
uuid: "65b7ce1ba0a44b25a58db9e94b90014b"
format_stamp: "Formatted at 2026-08-06 11:36:36 on dist-test-slave-7c35"
server_key: "d82b1c0ededfa2bf8aff3c72104fae12"
server_key_iv: "b650671a18ee3a237f1ed35c54d39624"
server_key_version: "encryptionkey@0"
I20260806 11:36:37.014217 26817 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786016169045816-21563-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786016169045816-21563-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786016169045816-21563-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:36:37.019423 26817 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:36:37.020391 26817 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:36:37.020689 26817 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:36:37.021518 26817 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:36:37.022608 26817 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:36:37.022689 26817 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:37.022768 26817 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:36:37.022812 26817 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:37.032888 26817 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.195:39575
I20260806 11:36:37.032910 26951 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.195:39575 every 8 connection(s)
I20260806 11:36:37.033911 26817 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
I20260806 11:36:37.034320 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 26817
I20260806 11:36:37.034469 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786016169045816-21563-0/minicluster-data/ts-2/wal/instance
I20260806 11:36:37.034821 21563 external_mini_cluster.cc:1428] Setting key f2013624f4f58895a0d516583a658438
I20260806 11:36:37.041352 26520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:37.036381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:52997 (local address 127.21.14.254:33271)
0806 11:36:37.036557 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:37.036562 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:37.037919 (+ 1357us) server_negotiation.cc:407] Connection header received
0806 11:36:37.039344 (+ 1425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:37.039347 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:37.039387 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:37.039492 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:37.040785 (+ 1293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:37.040788 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:37.040791 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 11:36:37.040805 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:37.040827 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 11:36:37.040836 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:37.040934 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:37.041186 (+ 252us) server_negotiation.cc:299] Negotiation successful
0806 11:36:37.041232 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 11:36:37.042418 26954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:37.036903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:33271 (local address 127.21.14.195:52997)
0806 11:36:37.037734 (+ 831us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:37.037794 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:37.039126 (+ 1332us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:37.039727 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:37.039736 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:37.039837 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:37.040454 (+ 617us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 11:36:37.040456 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:37.040505 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 11:36:37.040587 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:37.040992 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:37.041002 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:37.041008 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 11:36:37.041211 (+ 203us) client_negotiation.cc:240] Negotiation successful
0806 11:36:37.041449 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":159,"threads_started":1}
I20260806 11:36:37.043859 26952 heartbeater.cc:347] Connected to a master server at 127.21.14.254:33271
I20260806 11:36:37.044157 26952 heartbeater.cc:464] Registering TS with master...
I20260806 11:36:37.044661 26952 heartbeater.cc:511] Master 127.21.14.254:33271 requested a full tablet report, sending...
I20260806 11:36:37.046011 26481 ts_manager.cc:194] Registered new tserver with Master: 65b7ce1ba0a44b25a58db9e94b90014b (127.21.14.195:39575)
I20260806 11:36:37.049357 21563 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 11:36:37.053612 26520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:37.051565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51306 (local address 127.21.14.254:33271)
0806 11:36:37.051706 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:37.051709 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:37.052076 (+ 367us) server_negotiation.cc:407] Connection header received
0806 11:36:37.052196 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:37.052199 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:37.052238 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:37.052316 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:37.053238 (+ 922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:37.053242 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:37.053244 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 11:36:37.053256 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:37.053276 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 11:36:37.053285 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:37.053369 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:37.053461 (+ 92us) server_negotiation.cc:299] Negotiation successful
0806 11:36:37.053500 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 11:36:37.057045 26481 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51306:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 11:36:37.060348 26481 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 11:36:37.075132 26964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:37.070983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.193:46069 (local address 127.0.0.1:44222)
0806 11:36:37.071638 (+ 655us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:37.071699 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:37.071855 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:37.073069 (+ 1214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:37.073075 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:37.073143 (+ 68us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:37.073842 (+ 699us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 11:36:37.073844 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:37.073868 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 11:36:37.073943 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:37.074701 (+ 758us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:37.074708 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:37.074712 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 11:36:37.074809 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 11:36:37.074879 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":98,"threads_started":1}
I20260806 11:36:37.075244 26966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:37.071215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32946 (local address 127.21.14.195:39575)
0806 11:36:37.072859 (+ 1644us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:37.072867 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:37.072981 (+ 114us) server_negotiation.cc:407] Connection header received
0806 11:36:37.073059 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:37.073072 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:37.073240 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:37.073361 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:37.074307 (+ 946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:37.074314 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:37.074326 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 11:36:37.074338 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:37.074385 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 11:36:37.074422 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:37.074529 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:37.074977 (+ 448us) server_negotiation.cc:299] Negotiation successful
0806 11:36:37.075080 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1546,"thread_start_us":207,"threads_started":1}
I20260806 11:36:37.075609 26965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:37.071097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44222 (local address 127.21.14.193:46069)
0806 11:36:37.071862 (+ 765us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:37.071877 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:37.071913 (+ 36us) server_negotiation.cc:407] Connection header received
0806 11:36:37.071976 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:37.071982 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:37.072214 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:37.072371 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:37.074115 (+ 1744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:37.074122 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:37.074133 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 11:36:37.074144 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:37.074214 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 11:36:37.074258 (+ 44us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:37.074387 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:37.075013 (+ 626us) server_negotiation.cc:299] Negotiation successful
0806 11:36:37.075079 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":644,"thread_start_us":105,"threads_started":1}
I20260806 11:36:37.075944 26967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:37.071660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.195:39575 (local address 127.0.0.1:32946)
0806 11:36:37.072372 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:37.072387 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:37.072461 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:37.073396 (+ 935us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:37.073402 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:37.073421 (+ 19us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:37.074116 (+ 695us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 11:36:37.074118 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:37.074128 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 11:36:37.074157 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:37.074583 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:37.074598 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:37.074607 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 11:36:37.074734 (+ 127us) client_negotiation.cc:240] Negotiation successful
0806 11:36:37.074811 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":647,"thread_start_us":108,"threads_started":1}
I20260806 11:36:37.076792 26968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:37.071919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.194:39973 (local address 127.0.0.1:47210)
0806 11:36:37.073561 (+ 1642us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:37.073577 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:37.073704 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:37.074254 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:37.074257 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:37.074274 (+ 17us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:37.075969 (+ 1695us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 11:36:37.075971 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:37.075984 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 11:36:37.076046 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:37.076496 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:37.076500 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:37.076504 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 11:36:37.076599 (+ 95us) client_negotiation.cc:240] Negotiation successful
0806 11:36:37.076647 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1548,"thread_start_us":116,"threads_started":1}
I20260806 11:36:37.076880 26969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:37.072241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47210 (local address 127.21.14.194:39973)
0806 11:36:37.072658 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:37.072667 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:37.073827 (+ 1160us) server_negotiation.cc:407] Connection header received
0806 11:36:37.073892 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:37.073898 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:37.074100 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:37.074255 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:37.076256 (+ 2001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:37.076263 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:37.076275 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 11:36:37.076288 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:37.076342 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 11:36:37.076372 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:37.076524 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:37.076623 (+ 99us) server_negotiation.cc:299] Negotiation successful
0806 11:36:37.076709 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":118,"threads_started":1}
I20260806 11:36:37.079286 26608 tablet_service.cc:1467] Processing CreateTablet for tablet 4aa80bfc3b684d369bf5dcf5e743d80c (DEFAULT_TABLE table=test-table [id=18857847a4874fa0bd6682b1a36ea5f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:36:37.079288 26883 tablet_service.cc:1467] Processing CreateTablet for tablet 4aa80bfc3b684d369bf5dcf5e743d80c (DEFAULT_TABLE table=test-table [id=18857847a4874fa0bd6682b1a36ea5f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:36:37.080616 26883 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4aa80bfc3b684d369bf5dcf5e743d80c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:37.080616 26608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4aa80bfc3b684d369bf5dcf5e743d80c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:37.080854 26746 tablet_service.cc:1467] Processing CreateTablet for tablet 4aa80bfc3b684d369bf5dcf5e743d80c (DEFAULT_TABLE table=test-table [id=18857847a4874fa0bd6682b1a36ea5f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:36:37.081851 26746 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4aa80bfc3b684d369bf5dcf5e743d80c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:37.090014 26971 tablet_bootstrap.cc:492] T 4aa80bfc3b684d369bf5dcf5e743d80c P 65b7ce1ba0a44b25a58db9e94b90014b: Bootstrap starting.
I20260806 11:36:37.090014 26972 tablet_bootstrap.cc:492] T 4aa80bfc3b684d369bf5dcf5e743d80c P be2c4aef62d7499d85c9a4f6eff03008: Bootstrap starting.
I20260806 11:36:37.091622 26970 tablet_bootstrap.cc:492] T 4aa80bfc3b684d369bf5dcf5e743d80c P 91c66f260f724672839f57beb0819074: Bootstrap starting.
I20260806 11:36:37.093034 26971 tablet_bootstrap.cc:654] T 4aa80bfc3b684d369bf5dcf5e743d80c P 65b7ce1ba0a44b25a58db9e94b90014b: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:37.093034 26972 tablet_bootstrap.cc:654] T 4aa80bfc3b684d369bf5dcf5e743d80c P be2c4aef62d7499d85c9a4f6eff03008: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:37.093971 26970 tablet_bootstrap.cc:654] T 4aa80bfc3b684d369bf5dcf5e743d80c P 91c66f260f724672839f57beb0819074: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:37.094120 26971 log.cc:824] T 4aa80bfc3b684d369bf5dcf5e743d80c P 65b7ce1ba0a44b25a58db9e94b90014b: Log is configured to *not* fsync() on all Append() calls
I20260806 11:36:37.094120 26972 log.cc:824] T 4aa80bfc3b684d369bf5dcf5e743d80c P be2c4aef62d7499d85c9a4f6eff03008: Log is configured to *not* fsync() on all Append() calls
I20260806 11:36:37.094913 26970 log.cc:824] T 4aa80bfc3b684d369bf5dcf5e743d80c P 91c66f260f724672839f57beb0819074: Log is configured to *not* fsync() on all Append() calls
I20260806 11:36:37.096771 26972 tablet_bootstrap.cc:492] T 4aa80bfc3b684d369bf5dcf5e743d80c P be2c4aef62d7499d85c9a4f6eff03008: No bootstrap required, opened a new log
I20260806 11:36:37.096861 26971 tablet_bootstrap.cc:492] T 4aa80bfc3b684d369bf5dcf5e743d80c P 65b7ce1ba0a44b25a58db9e94b90014b: No bootstrap required, opened a new log
I20260806 11:36:37.097047 26971 ts_tablet_manager.cc:1397] T 4aa80bfc3b684d369bf5dcf5e743d80c P 65b7ce1ba0a44b25a58db9e94b90014b: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260806 11:36:37.097048 26972 ts_tablet_manager.cc:1397] T 4aa80bfc3b684d369bf5dcf5e743d80c P be2c4aef62d7499d85c9a4f6eff03008: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 11:36:37.097129 26970 tablet_bootstrap.cc:492] T 4aa80bfc3b684d369bf5dcf5e743d80c P 91c66f260f724672839f57beb0819074: No bootstrap required, opened a new log
I20260806 11:36:37.097299 26970 ts_tablet_manager.cc:1397] T 4aa80bfc3b684d369bf5dcf5e743d80c P 91c66f260f724672839f57beb0819074: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 11:36:37.101089 26970 raft_consensus.cc:348] T 4aa80bfc3b684d369bf5dcf5e743d80c P 91c66f260f724672839f57beb0819074 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be2c4aef62d7499d85c9a4f6eff03008" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 46069 } } peers { permanent_uuid: "65b7ce1ba0a44b25a58db9e94b90014b" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 39575 } } peers { permanent_uuid: "91c66f260f724672839f57beb0819074" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 39973 } }
I20260806 11:36:37.101413 26970 raft_consensus.cc:374] T 4aa80bfc3b684d369bf5dcf5e743d80c P 91c66f260f724672839f57beb0819074 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:37.101480 26970 raft_consensus.cc:729] T 4aa80bfc3b684d369bf5dcf5e743d80c P 91c66f260f724672839f57beb0819074 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91c66f260f724672839f57beb0819074, State: Initialized, Role: FOLLOWER
I20260806 11:36:37.102344 26970 consensus_queue.cc:260] T 4aa80bfc3b684d369bf5dcf5e743d80c P 91c66f260f724672839f57beb0819074 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be2c4aef62d7499d85c9a4f6eff03008" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 46069 } } peers { permanent_uuid: "65b7ce1ba0a44b25a58db9e94b90014b" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 39575 } } peers { permanent_uuid: "91c66f260f724672839f57beb0819074" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 39973 } }
I20260806 11:36:37.102362 26971 raft_consensus.cc:348] T 4aa80bfc3b684d369bf5dcf5e743d80c P 65b7ce1ba0a44b25a58db9e94b90014b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be2c4aef62d7499d85c9a4f6eff03008" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 46069 } } peers { permanent_uuid: "65b7ce1ba0a44b25a58db9e94b90014b" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 39575 } } peers { permanent_uuid: "91c66f260f724672839f57beb0819074" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 39973 } }
I20260806 11:36:37.102362 26972 raft_consensus.cc:348] T 4aa80bfc3b684d369bf5dcf5e743d80c P be2c4aef62d7499d85c9a4f6eff03008 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be2c4aef62d7499d85c9a4f6eff03008" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 46069 } } peers { permanent_uuid: "65b7ce1ba0a44b25a58db9e94b90014b" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 39575 } } peers { permanent_uuid: "91c66f260f724672839f57beb0819074" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 39973 } }
I20260806 11:36:37.102999 26972 raft_consensus.cc:374] T 4aa80bfc3b684d369bf5dcf5e743d80c P be2c4aef62d7499d85c9a4f6eff03008 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:37.102999 26971 raft_consensus.cc:374] T 4aa80bfc3b684d369bf5dcf5e743d80c P 65b7ce1ba0a44b25a58db9e94b90014b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:37.103076 26972 raft_consensus.cc:729] T 4aa80bfc3b684d369bf5dcf5e743d80c P be2c4aef62d7499d85c9a4f6eff03008 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be2c4aef62d7499d85c9a4f6eff03008, State: Initialized, Role: FOLLOWER
I20260806 11:36:37.103077 26971 raft_consensus.cc:729] T 4aa80bfc3b684d369bf5dcf5e743d80c P 65b7ce1ba0a44b25a58db9e94b90014b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65b7ce1ba0a44b25a58db9e94b90014b, State: Initialized, Role: FOLLOWER
I20260806 11:36:37.103633 26970 ts_tablet_manager.cc:1428] T 4aa80bfc3b684d369bf5dcf5e743d80c P 91c66f260f724672839f57beb0819074: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 11:36:37.103677 26814 heartbeater.cc:503] Master 127.21.14.254:33271 was elected leader, sending a full tablet report...
I20260806 11:36:37.103667 26971 consensus_queue.cc:260] T 4aa80bfc3b684d369bf5dcf5e743d80c P 65b7ce1ba0a44b25a58db9e94b90014b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be2c4aef62d7499d85c9a4f6eff03008" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 46069 } } peers { permanent_uuid: "65b7ce1ba0a44b25a58db9e94b90014b" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 39575 } } peers { permanent_uuid: "91c66f260f724672839f57beb0819074" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 39973 } }
I20260806 11:36:37.104511 26952 heartbeater.cc:503] Master 127.21.14.254:33271 was elected leader, sending a full tablet report...
I20260806 11:36:37.105180 26971 ts_tablet_manager.cc:1428] T 4aa80bfc3b684d369bf5dcf5e743d80c P 65b7ce1ba0a44b25a58db9e94b90014b: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260806 11:36:37.105680 26972 consensus_queue.cc:260] T 4aa80bfc3b684d369bf5dcf5e743d80c P be2c4aef62d7499d85c9a4f6eff03008 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be2c4aef62d7499d85c9a4f6eff03008" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 46069 } } peers { permanent_uuid: "65b7ce1ba0a44b25a58db9e94b90014b" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 39575 } } peers { permanent_uuid: "91c66f260f724672839f57beb0819074" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 39973 } }
I20260806 11:36:37.106720 26676 heartbeater.cc:503] Master 127.21.14.254:33271 was elected leader, sending a full tablet report...
I20260806 11:36:37.106971 26972 ts_tablet_manager.cc:1428] T 4aa80bfc3b684d369bf5dcf5e743d80c P be2c4aef62d7499d85c9a4f6eff03008: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
W20260806 11:36:37.145783 26679 tablet.cc:2369] T 4aa80bfc3b684d369bf5dcf5e743d80c P be2c4aef62d7499d85c9a4f6eff03008: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 11:36:37.183882 26977 raft_consensus.cc:482] T 4aa80bfc3b684d369bf5dcf5e743d80c P 65b7ce1ba0a44b25a58db9e94b90014b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 11:36:37.184124 26977 raft_consensus.cc:504] T 4aa80bfc3b684d369bf5dcf5e743d80c P 65b7ce1ba0a44b25a58db9e94b90014b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be2c4aef62d7499d85c9a4f6eff03008" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 46069 } } peers { permanent_uuid: "65b7ce1ba0a44b25a58db9e94b90014b" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 39575 } } peers { permanent_uuid: "91c66f260f724672839f57beb0819074" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 39973 } }
I20260806 11:36:37.185461 26977 leader_election.cc:288] T 4aa80bfc3b684d369bf5dcf5e743d80c P 65b7ce1ba0a44b25a58db9e94b90014b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be2c4aef62d7499d85c9a4f6eff03008 (127.21.14.193:46069), 91c66f260f724672839f57beb0819074 (127.21.14.194:39973)
I20260806 11:36:37.188675 26954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:37.185902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.193:46069 (local address 127.21.14.195:47863)
0806 11:36:37.186271 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:37.186287 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:37.186569 (+ 282us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:37.186959 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:37.186962 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:37.186979 (+ 17us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:37.187897 (+ 918us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 11:36:37.187898 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:37.187912 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 11:36:37.187942 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:37.188372 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:37.188376 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:37.188382 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 11:36:37.188489 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 11:36:37.188532 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274}
I20260806 11:36:37.188726 26965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:37.186399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:47863 (local address 127.21.14.193:46069)
0806 11:36:37.186670 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:37.186677 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:37.186697 (+ 20us) server_negotiation.cc:407] Connection header received
0806 11:36:37.186756 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:37.186761 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:37.186802 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:37.186937 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:37.188171 (+ 1234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:37.188177 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:37.188180 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 11:36:37.188197 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:37.188225 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 11:36:37.188239 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:37.188349 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:37.188510 (+ 161us) server_negotiation.cc:299] Negotiation successful
0806 11:36:37.188579 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145}
I20260806 11:36:37.189172 26969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:37.186897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:50441 (local address 127.21.14.194:39973)
0806 11:36:37.187069 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:37.187072 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:37.187480 (+ 408us) server_negotiation.cc:407] Connection header received
0806 11:36:37.187546 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:37.187549 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:37.187590 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:37.187687 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:37.188638 (+ 951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:37.188642 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:37.188645 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 11:36:37.188657 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:37.188679 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 11:36:37.188690 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:37.188788 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:37.188951 (+ 163us) server_negotiation.cc:299] Negotiation successful
0806 11:36:37.188996 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 11:36:37.189522 26980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:37.186691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.194:39973 (local address 127.21.14.195:50441)
0806 11:36:37.187155 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:37.187171 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:37.187419 (+ 248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:37.187883 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:37.187886 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:37.187902 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:37.188430 (+ 528us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 11:36:37.188435 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:37.188449 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 11:36:37.188479 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:37.188838 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:37.188842 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:37.188845 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 11:36:37.189361 (+ 516us) client_negotiation.cc:240] Negotiation successful
0806 11:36:37.189404 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":172,"threads_started":1}
I20260806 11:36:37.189550 26629 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4aa80bfc3b684d369bf5dcf5e743d80c" candidate_uuid: "65b7ce1ba0a44b25a58db9e94b90014b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be2c4aef62d7499d85c9a4f6eff03008" is_pre_election: true
I20260806 11:36:37.189994 26629 raft_consensus.cc:2454] T 4aa80bfc3b684d369bf5dcf5e743d80c P be2c4aef62d7499d85c9a4f6eff03008 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 65b7ce1ba0a44b25a58db9e94b90014b in term 0.
I20260806 11:36:37.190285 26767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4aa80bfc3b684d369bf5dcf5e743d80c" candidate_uuid: "65b7ce1ba0a44b25a58db9e94b90014b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91c66f260f724672839f57beb0819074" is_pre_election: true
I20260806 11:36:37.190670 26767 raft_consensus.cc:2454] T 4aa80bfc3b684d369bf5dcf5e743d80c P 91c66f260f724672839f57beb0819074 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 65b7ce1ba0a44b25a58db9e94b90014b in term 0.
I20260806 11:36:37.190685 26834 leader_election.cc:302] T 4aa80bfc3b684d369bf5dcf5e743d80c P 65b7ce1ba0a44b25a58db9e94b90014b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65b7ce1ba0a44b25a58db9e94b90014b, be2c4aef62d7499d85c9a4f6eff03008; no voters:
I20260806 11:36:37.191114 26977 raft_consensus.cc:2781] T 4aa80bfc3b684d369bf5dcf5e743d80c P 65b7ce1ba0a44b25a58db9e94b90014b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 11:36:37.191201 26977 raft_consensus.cc:482] T 4aa80bfc3b684d369bf5dcf5e743d80c P 65b7ce1ba0a44b25a58db9e94b90014b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 11:36:37.191243 26977 raft_consensus.cc:3037] T 4aa80bfc3b684d369bf5dcf5e743d80c P 65b7ce1ba0a44b25a58db9e94b90014b [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:36:37.192726 26977 raft_consensus.cc:504] T 4aa80bfc3b684d369bf5dcf5e743d80c P 65b7ce1ba0a44b25a58db9e94b90014b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be2c4aef62d7499d85c9a4f6eff03008" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 46069 } } peers { permanent_uuid: "65b7ce1ba0a44b25a58db9e94b90014b" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 39575 } } peers { permanent_uuid: "91c66f260f724672839f57beb0819074" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 39973 } }
I20260806 11:36:37.193143 26977 leader_election.cc:288] T 4aa80bfc3b684d369bf5dcf5e743d80c P 65b7ce1ba0a44b25a58db9e94b90014b [CANDIDATE]: Term 1 election: Requested vote from peers be2c4aef62d7499d85c9a4f6eff03008 (127.21.14.193:46069), 91c66f260f724672839f57beb0819074 (127.21.14.194:39973)
I20260806 11:36:37.193820 26629 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4aa80bfc3b684d369bf5dcf5e743d80c" candidate_uuid: "65b7ce1ba0a44b25a58db9e94b90014b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be2c4aef62d7499d85c9a4f6eff03008"
I20260806 11:36:37.193828 26767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4aa80bfc3b684d369bf5dcf5e743d80c" candidate_uuid: "65b7ce1ba0a44b25a58db9e94b90014b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91c66f260f724672839f57beb0819074"
I20260806 11:36:37.193998 26767 raft_consensus.cc:3037] T 4aa80bfc3b684d369bf5dcf5e743d80c P 91c66f260f724672839f57beb0819074 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:36:37.193962 26629 raft_consensus.cc:3037] T 4aa80bfc3b684d369bf5dcf5e743d80c P be2c4aef62d7499d85c9a4f6eff03008 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:36:37.195304 26767 raft_consensus.cc:2454] T 4aa80bfc3b684d369bf5dcf5e743d80c P 91c66f260f724672839f57beb0819074 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 65b7ce1ba0a44b25a58db9e94b90014b in term 1.
I20260806 11:36:37.195559 26629 raft_consensus.cc:2454] T 4aa80bfc3b684d369bf5dcf5e743d80c P be2c4aef62d7499d85c9a4f6eff03008 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 65b7ce1ba0a44b25a58db9e94b90014b in term 1.
I20260806 11:36:37.195721 26834 leader_election.cc:302] T 4aa80bfc3b684d369bf5dcf5e743d80c P 65b7ce1ba0a44b25a58db9e94b90014b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65b7ce1ba0a44b25a58db9e94b90014b, 91c66f260f724672839f57beb0819074; no voters:
I20260806 11:36:37.195972 26977 raft_consensus.cc:2781] T 4aa80bfc3b684d369bf5dcf5e743d80c P 65b7ce1ba0a44b25a58db9e94b90014b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:36:37.196235 26977 raft_consensus.cc:686] T 4aa80bfc3b684d369bf5dcf5e743d80c P 65b7ce1ba0a44b25a58db9e94b90014b [term 1 LEADER]: Becoming Leader. State: Replica: 65b7ce1ba0a44b25a58db9e94b90014b, State: Running, Role: LEADER
I20260806 11:36:37.196728 26977 consensus_queue.cc:237] T 4aa80bfc3b684d369bf5dcf5e743d80c P 65b7ce1ba0a44b25a58db9e94b90014b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be2c4aef62d7499d85c9a4f6eff03008" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 46069 } } peers { permanent_uuid: "65b7ce1ba0a44b25a58db9e94b90014b" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 39575 } } peers { permanent_uuid: "91c66f260f724672839f57beb0819074" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 39973 } }
I20260806 11:36:37.201395 26480 catalog_manager.cc:5495] T 4aa80bfc3b684d369bf5dcf5e743d80c P 65b7ce1ba0a44b25a58db9e94b90014b reported cstate change: term changed from 0 to 1, leader changed from <none> to 65b7ce1ba0a44b25a58db9e94b90014b (127.21.14.195). New cstate: current_term: 1 leader_uuid: "65b7ce1ba0a44b25a58db9e94b90014b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be2c4aef62d7499d85c9a4f6eff03008" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 46069 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "65b7ce1ba0a44b25a58db9e94b90014b" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 39575 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91c66f260f724672839f57beb0819074" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 39973 } health_report { overall_health: UNKNOWN } } }
I20260806 11:36:37.236732 26966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:37.234504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32950 (local address 127.21.14.195:39575)
0806 11:36:37.234710 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:37.234714 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:37.234732 (+ 18us) server_negotiation.cc:407] Connection header received
0806 11:36:37.234828 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:37.234830 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:37.234938 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:37.235028 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:37.236069 (+ 1041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:37.236073 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:37.236075 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 11:36:37.236088 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:37.236111 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 11:36:37.236120 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:37.236224 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:37.236401 (+ 177us) server_negotiation.cc:299] Negotiation successful
0806 11:36:37.236531 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260806 11:36:37.244701 26767 raft_consensus.cc:1261] T 4aa80bfc3b684d369bf5dcf5e743d80c P 91c66f260f724672839f57beb0819074 [term 1 FOLLOWER]: Refusing update from remote peer 65b7ce1ba0a44b25a58db9e94b90014b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 11:36:37.244701 26629 raft_consensus.cc:1261] T 4aa80bfc3b684d369bf5dcf5e743d80c P be2c4aef62d7499d85c9a4f6eff03008 [term 1 FOLLOWER]: Refusing update from remote peer 65b7ce1ba0a44b25a58db9e94b90014b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 11:36:37.245788 26977 consensus_queue.cc:1035] T 4aa80bfc3b684d369bf5dcf5e743d80c P 65b7ce1ba0a44b25a58db9e94b90014b [LEADER]: Connected to new peer: Peer: permanent_uuid: "be2c4aef62d7499d85c9a4f6eff03008" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 46069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 11:36:37.246129 26981 consensus_queue.cc:1035] T 4aa80bfc3b684d369bf5dcf5e743d80c P 65b7ce1ba0a44b25a58db9e94b90014b [LEADER]: Connected to new peer: Peer: permanent_uuid: "91c66f260f724672839f57beb0819074" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 39973 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 11:36:37.257706 26986 mvcc.cc:205] Tried to move back new op lower bound from 7315522343902289920 to 7315522343722041344. Current Snapshot: MvccSnapshot[applied={T|T < 7315522343902289920}]
I20260806 11:36:37.258423 26985 mvcc.cc:205] Tried to move back new op lower bound from 7315522343902289920 to 7315522343722041344. Current Snapshot: MvccSnapshot[applied={T|T < 7315522343902289920}]
I20260806 11:36:37.267645 26480 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51306:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 11:36:37.268003 26480 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51306:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 11:36:37.271426 26480 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8d6466c9e62046d1a3d5ca9b20f8b569: Sending DeleteTablet for 3 replicas of tablet 4aa80bfc3b684d369bf5dcf5e743d80c
I20260806 11:36:37.272557 26746 tablet_service.cc:1514] Processing DeleteTablet for tablet 4aa80bfc3b684d369bf5dcf5e743d80c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:36:37 UTC) from {username='slave'} at 127.0.0.1:47210
I20260806 11:36:37.272652 26883 tablet_service.cc:1514] Processing DeleteTablet for tablet 4aa80bfc3b684d369bf5dcf5e743d80c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:36:37 UTC) from {username='slave'} at 127.0.0.1:32946
I20260806 11:36:37.272720 26608 tablet_service.cc:1514] Processing DeleteTablet for tablet 4aa80bfc3b684d369bf5dcf5e743d80c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:36:37 UTC) from {username='slave'} at 127.0.0.1:44222
I20260806 11:36:37.273376 26995 tablet_replica.cc:330] T 4aa80bfc3b684d369bf5dcf5e743d80c P be2c4aef62d7499d85c9a4f6eff03008: stopping tablet replica
I20260806 11:36:37.273590 26994 tablet_replica.cc:330] T 4aa80bfc3b684d369bf5dcf5e743d80c P 91c66f260f724672839f57beb0819074: stopping tablet replica
I20260806 11:36:37.273820 26995 raft_consensus.cc:2229] T 4aa80bfc3b684d369bf5dcf5e743d80c P be2c4aef62d7499d85c9a4f6eff03008 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 11:36:37.273905 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 26541
I20260806 11:36:37.274019 26994 raft_consensus.cc:2229] T 4aa80bfc3b684d369bf5dcf5e743d80c P 91c66f260f724672839f57beb0819074 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 11:36:37.274271 26995 raft_consensus.cc:2258] T 4aa80bfc3b684d369bf5dcf5e743d80c P be2c4aef62d7499d85c9a4f6eff03008 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 11:36:37.274838 26996 tablet_replica.cc:330] T 4aa80bfc3b684d369bf5dcf5e743d80c P 65b7ce1ba0a44b25a58db9e94b90014b: stopping tablet replica
I20260806 11:36:37.275223 26996 raft_consensus.cc:2229] T 4aa80bfc3b684d369bf5dcf5e743d80c P 65b7ce1ba0a44b25a58db9e94b90014b [term 1 LEADER]: Raft consensus shutting down.
I20260806 11:36:37.275476 26994 raft_consensus.cc:2258] T 4aa80bfc3b684d369bf5dcf5e743d80c P 91c66f260f724672839f57beb0819074 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 11:36:37.275691 26996 raft_consensus.cc:2258] T 4aa80bfc3b684d369bf5dcf5e743d80c P 65b7ce1ba0a44b25a58db9e94b90014b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 11:36:37.277796 26994 ts_tablet_manager.cc:1905] T 4aa80bfc3b684d369bf5dcf5e743d80c P 91c66f260f724672839f57beb0819074: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 11:36:37.281339 26996 ts_tablet_manager.cc:1905] T 4aa80bfc3b684d369bf5dcf5e743d80c P 65b7ce1ba0a44b25a58db9e94b90014b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 11:36:37.281649 26994 ts_tablet_manager.cc:1918] T 4aa80bfc3b684d369bf5dcf5e743d80c P 91c66f260f724672839f57beb0819074: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 11:36:37.281845 26994 log.cc:1196] T 4aa80bfc3b684d369bf5dcf5e743d80c P 91c66f260f724672839f57beb0819074: Deleting WAL directory at /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786016169045816-21563-0/minicluster-data/ts-1/wal/wals/4aa80bfc3b684d369bf5dcf5e743d80c
I20260806 11:36:37.282274 26994 ts_tablet_manager.cc:1939] T 4aa80bfc3b684d369bf5dcf5e743d80c P 91c66f260f724672839f57beb0819074: Deleting consensus metadata
I20260806 11:36:37.284004 26466 catalog_manager.cc:4841] TS 91c66f260f724672839f57beb0819074 (127.21.14.194:39973): tablet 4aa80bfc3b684d369bf5dcf5e743d80c (table test-table [id=18857847a4874fa0bd6682b1a36ea5f2]) successfully deleted
I20260806 11:36:37.284582 26996 ts_tablet_manager.cc:1918] T 4aa80bfc3b684d369bf5dcf5e743d80c P 65b7ce1ba0a44b25a58db9e94b90014b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 11:36:37.284719 26996 log.cc:1196] T 4aa80bfc3b684d369bf5dcf5e743d80c P 65b7ce1ba0a44b25a58db9e94b90014b: Deleting WAL directory at /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786016169045816-21563-0/minicluster-data/ts-2/wal/wals/4aa80bfc3b684d369bf5dcf5e743d80c
I20260806 11:36:37.285116 26996 ts_tablet_manager.cc:1939] T 4aa80bfc3b684d369bf5dcf5e743d80c P 65b7ce1ba0a44b25a58db9e94b90014b: Deleting consensus metadata
I20260806 11:36:37.286363 26465 catalog_manager.cc:4841] TS 65b7ce1ba0a44b25a58db9e94b90014b (127.21.14.195:39575): tablet 4aa80bfc3b684d369bf5dcf5e743d80c (table test-table [id=18857847a4874fa0bd6682b1a36ea5f2]) successfully deleted
W20260806 11:36:37.288378 26466 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.14.193:46069 (error 108)
I20260806 11:36:37.289443 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 26681
I20260806 11:36:37.289682 26967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:37.289255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.193:46069 (local address 127.0.0.1:44224)
0806 11:36:37.289422 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:37.289569 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.193:46069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260806 11:36:37.289883 26466 catalog_manager.cc:4584] TS be2c4aef62d7499d85c9a4f6eff03008 (127.21.14.193:46069): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4aa80bfc3b684d369bf5dcf5e743d80c: Network error: Client connection negotiation failed: client connection to 127.21.14.193:46069: connect: Connection refused (error 111)
I20260806 11:36:37.300570 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 26817
I20260806 11:36:37.308995 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 26449
2026-08-06T11:36:37Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1174 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260806 11:36:37.336513 21563 mini_oidc.cc:152] Starting JWKS server
I20260806 11:36:37.338366 21563 webserver.cc:460] Webserver started at https://127.0.0.1:33017/ using document root <none> and password file <none>
I20260806 11:36:37.338574 21563 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 11:36:37.338939 21563 webserver.cc:460] Webserver started at http://127.0.0.1:44283/ using document root <none> and password file <none>
2026-08-06T11:36:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T11:36:37Z Disabled control of system clock
I20260806 11:36:37.351076 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786016169045816-21563-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786016169045816-21563-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786016169045816-21563-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:41817
--webserver_interface=127.21.14.254
--webserver_port=0
--builtin_ntp_servers=127.21.14.212:36593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.14.254:41817
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33017/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786016169045816-21563-0/testchainca.cert with env {}
W20260806 11:36:37.475703 27003 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:36:37.476051 27003 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:36:37.476097 27003 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:36:37.480662 27003 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 11:36:37.480746 27003 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:36:37.480762 27003 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:36:37.480779 27003 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 11:36:37.480795 27003 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 11:36:37.480813 27003 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 11:36:37.480826 27003 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33017/jwks.json
I20260806 11:36:37.486701 27003 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:36593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786016169045816-21563-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786016169045816-21563-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.14.254:41817
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:41817
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33017/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.14.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786016169045816-21563-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27003
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786016169045816-21563-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:36:37.487989 27003 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:36:37.489168 27003 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:36:37.498469 27010 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:37.498463 27011 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:37.498538 27003 server_base.cc:1034] running on GCE node
W20260806 11:36:37.498546 27013 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:37.499408 27003 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:36:37.500644 27003 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:36:37.501832 27003 hybrid_clock.cc:648] HybridClock initialized: now 1786016197501814 us; error 33 us; skew 500 ppm
I20260806 11:36:37.503957 27003 webserver.cc:460] Webserver started at http://127.21.14.254:45665/ using document root <none> and password file <none>
I20260806 11:36:37.504611 27003 fs_manager.cc:359] Metadata directory not provided
I20260806 11:36:37.504679 27003 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:36:37.504904 27003 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:36:37.507238 27003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786016169045816-21563-0/minicluster-data/master-0/data/instance:
uuid: "6ce3c3d66a6b43eaa6fee438b5c65b15"
format_stamp: "Formatted at 2026-08-06 11:36:37 on dist-test-slave-7c35"
server_key: "ba456ec9f7b8ca435f3ce5e486536dae"
server_key_iv: "bc565e6eb80ec91de4bfcf81788f948a"
server_key_version: "encryptionkey@0"
I20260806 11:36:37.507872 27003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786016169045816-21563-0/minicluster-data/master-0/wal/instance:
uuid: "6ce3c3d66a6b43eaa6fee438b5c65b15"
format_stamp: "Formatted at 2026-08-06 11:36:37 on dist-test-slave-7c35"
server_key: "ba456ec9f7b8ca435f3ce5e486536dae"
server_key_iv: "bc565e6eb80ec91de4bfcf81788f948a"
server_key_version: "encryptionkey@0"
I20260806 11:36:37.512205 27003 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 11:36:37.515204 27019 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:37.516695 27003 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 11:36:37.516906 27003 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786016169045816-21563-0/minicluster-data/master-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786016169045816-21563-0/minicluster-data/master-0/wal
uuid: "6ce3c3d66a6b43eaa6fee438b5c65b15"
format_stamp: "Formatted at 2026-08-06 11:36:37 on dist-test-slave-7c35"
server_key: "ba456ec9f7b8ca435f3ce5e486536dae"
server_key_iv: "bc565e6eb80ec91de4bfcf81788f948a"
server_key_version: "encryptionkey@0"
I20260806 11:36:37.517062 27003 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786016169045816-21563-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786016169045816-21563-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786016169045816-21563-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:36:37.538125 27003 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:36:37.539060 27003 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:36:37.539330 27003 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:36:37.548055 27003 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.254:41817
I20260806 11:36:37.548120 27074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.254:41817 every 8 connection(s)
I20260806 11:36:37.549207 27003 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
I20260806 11:36:37.549665 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 27003
I20260806 11:36:37.549798 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786016169045816-21563-0/minicluster-data/master-0/wal/instance
I20260806 11:36:37.550180 21563 external_mini_cluster.cc:1428] Setting key 906f44e3dd92e0697516cfceac794784
I20260806 11:36:37.554096 27075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:37.561947 27078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:37.551725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39954 (local address 127.21.14.254:41817)
0806 11:36:37.552568 (+ 843us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:37.552587 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:37.552631 (+ 44us) server_negotiation.cc:407] Connection header received
0806 11:36:37.553327 (+ 696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:37.553346 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:37.553820 (+ 474us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:37.554161 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:37.554714 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:37.555840 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:37.557496 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:37.558054 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:37.558218 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:37.559404 (+ 1186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:37.559418 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:37.559433 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 11:36:37.560102 (+ 669us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:37.560176 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 11:36:37.560199 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:37.560302 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:37.560542 (+ 240us) server_negotiation.cc:299] Negotiation successful
0806 11:36:37.561012 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":197,"threads_started":1}
I20260806 11:36:37.562454 27075 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ce3c3d66a6b43eaa6fee438b5c65b15: Bootstrap starting.
I20260806 11:36:37.565495 27075 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6ce3c3d66a6b43eaa6fee438b5c65b15: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:37.565717 21563 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 11:36:37.566387 27075 log.cc:824] T 00000000000000000000000000000000 P 6ce3c3d66a6b43eaa6fee438b5c65b15: Log is configured to *not* fsync() on all Append() calls
I20260806 11:36:37.568897 27075 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ce3c3d66a6b43eaa6fee438b5c65b15: No bootstrap required, opened a new log
I20260806 11:36:37.572933 27075 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6ce3c3d66a6b43eaa6fee438b5c65b15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ce3c3d66a6b43eaa6fee438b5c65b15" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 41817 } }
I20260806 11:36:37.573218 27075 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6ce3c3d66a6b43eaa6fee438b5c65b15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:37.573259 27075 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6ce3c3d66a6b43eaa6fee438b5c65b15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ce3c3d66a6b43eaa6fee438b5c65b15, State: Initialized, Role: FOLLOWER
I20260806 11:36:37.573680 27075 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ce3c3d66a6b43eaa6fee438b5c65b15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ce3c3d66a6b43eaa6fee438b5c65b15" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 41817 } }
I20260806 11:36:37.573807 27075 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6ce3c3d66a6b43eaa6fee438b5c65b15 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 11:36:37.573886 27075 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6ce3c3d66a6b43eaa6fee438b5c65b15 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 11:36:37.574002 27075 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6ce3c3d66a6b43eaa6fee438b5c65b15 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:36:37.575383 27075 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6ce3c3d66a6b43eaa6fee438b5c65b15 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ce3c3d66a6b43eaa6fee438b5c65b15" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 41817 } }
I20260806 11:36:37.575767 27075 leader_election.cc:302] T 00000000000000000000000000000000 P 6ce3c3d66a6b43eaa6fee438b5c65b15 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6ce3c3d66a6b43eaa6fee438b5c65b15; no voters:
I20260806 11:36:37.576280 27075 leader_election.cc:288] T 00000000000000000000000000000000 P 6ce3c3d66a6b43eaa6fee438b5c65b15 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 11:36:37.576328 27080 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6ce3c3d66a6b43eaa6fee438b5c65b15 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:36:37.576671 27080 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6ce3c3d66a6b43eaa6fee438b5c65b15 [term 1 LEADER]: Becoming Leader. State: Replica: 6ce3c3d66a6b43eaa6fee438b5c65b15, State: Running, Role: LEADER
I20260806 11:36:37.577047 27080 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6ce3c3d66a6b43eaa6fee438b5c65b15 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ce3c3d66a6b43eaa6fee438b5c65b15" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 41817 } }
I20260806 11:36:37.577778 27075 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6ce3c3d66a6b43eaa6fee438b5c65b15 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 11:36:37.578949 27081 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ce3c3d66a6b43eaa6fee438b5c65b15 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6ce3c3d66a6b43eaa6fee438b5c65b15" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ce3c3d66a6b43eaa6fee438b5c65b15" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 41817 } } }
I20260806 11:36:37.579110 27081 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ce3c3d66a6b43eaa6fee438b5c65b15 [sys.catalog]: This master's current role is: LEADER
I20260806 11:36:37.579388 27082 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ce3c3d66a6b43eaa6fee438b5c65b15 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ce3c3d66a6b43eaa6fee438b5c65b15. Latest consensus state: current_term: 1 leader_uuid: "6ce3c3d66a6b43eaa6fee438b5c65b15" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ce3c3d66a6b43eaa6fee438b5c65b15" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 41817 } } }
I20260806 11:36:37.579485 27082 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ce3c3d66a6b43eaa6fee438b5c65b15 [sys.catalog]: This master's current role is: LEADER
I20260806 11:36:37.579595 27089 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 11:36:37.583050 27089 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 11:36:37.589665 27089 catalog_manager.cc:1347] Generated new cluster ID: 0bc59f0e83654fb6a0a0da927f27b8ea
I20260806 11:36:37.589768 27089 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 11:36:37.600315 27089 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 11:36:37.601763 27089 catalog_manager.cc:1504] Loading token signing keys...
I20260806 11:36:37.608603 27089 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6ce3c3d66a6b43eaa6fee438b5c65b15: Generated new TSK 0
I20260806 11:36:37.609674 27089 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 11:36:37.653234 27106 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260806 11:36:37.653504 27106 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 11:36:37.654135 27078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:37.634287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39958 (local address 127.21.14.254:41817)
0806 11:36:37.634490 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:37.634494 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:37.634514 (+ 20us) server_negotiation.cc:407] Connection header received
0806 11:36:37.634812 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:37.634815 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:37.634983 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:37.635091 (+ 108us) server_negotiation.cc:226] Negotiated authn=JWT
0806 11:36:37.635496 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:37.636185 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:37.637566 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:37.637805 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:37.637914 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:37.638571 (+ 657us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 11:36:37.653734 (+ 15163us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33017/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 11:36:37.654017 (+ 283us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39958: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33017/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":103}
W20260806 11:36:37.654232 27078 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:39958: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33017/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260806 11:36:37.656068 27105 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:36:37.633830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:41817 (local address 127.0.0.1:39958)
0806 11:36:37.634363 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:37.634387 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:37.634548 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:37.635143 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:37.635146 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:37.635158 (+ 12us) client_negotiation.cc:189] Negotiated authn=JWT
0806 11:36:37.635311 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:37.635319 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:37.636333 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:37.636337 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:37.637392 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:37.637406 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:37.637976 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:37.637981 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:37.638111 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:37.638218 (+ 107us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 11:36:37.654029 (+ 15811us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33017/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 11:36:37.654374 (+ 345us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.21.14.254:41817: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33017/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":162,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.21.14.254:41817: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33017/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260806 11:36:37.656987 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 27003
2026-08-06T11:36:37Z chronyd exiting
I20260806 11:36:37.698064 21563 test_util.cc:175] -----------------------------------------------
I20260806 11:36:37.698230 21563 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786016169045816-21563-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (363 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260806 11:36:37.699551 21563 mini_oidc.cc:152] Starting JWKS server
I20260806 11:36:37.700610 21563 webserver.cc:460] Webserver started at https://127.0.0.1:34435/ using document root <none> and password file <none>
I20260806 11:36:37.700714 21563 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 11:36:37.700954 21563 webserver.cc:460] Webserver started at http://127.0.0.1:35055/ using document root <none> and password file <none>
2026-08-06T11:36:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T11:36:37Z Disabled control of system clock
I20260806 11:36:37.713442 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786016169045816-21563-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786016169045816-21563-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786016169045816-21563-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:42739
--webserver_interface=127.21.14.254
--webserver_port=0
--builtin_ntp_servers=127.21.14.212:42971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.14.254:42739
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34435/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786016169045816-21563-0/testchainca.cert with env {}
W20260806 11:36:37.839885 27112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:36:37.840224 27112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:36:37.840268 27112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:36:37.844481 27112 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 11:36:37.844555 27112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:36:37.844571 27112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:36:37.844589 27112 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 11:36:37.844604 27112 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 11:36:37.844621 27112 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 11:36:37.844635 27112 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34435/jwks.json
I20260806 11:36:37.850337 27112 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:42971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786016169045816-21563-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786016169045816-21563-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.14.254:42739
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:42739
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34435/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.14.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786016169045816-21563-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27112
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786016169045816-21563-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:36:37.851635 27112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:36:37.852659 27112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 11:36:37.861773 27112 server_base.cc:1034] running on GCE node
W20260806 11:36:37.861701 27118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:37.861707 27120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:37.861701 27117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:37.862641 27112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:36:37.863868 27112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:36:37.865077 27112 hybrid_clock.cc:648] HybridClock initialized: now 1786016197865042 us; error 47 us; skew 500 ppm
I20260806 11:36:37.867166 27112 webserver.cc:460] Webserver started at http://127.21.14.254:35383/ using document root <none> and password file <none>
I20260806 11:36:37.867796 27112 fs_manager.cc:359] Metadata directory not provided
I20260806 11:36:37.867863 27112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:36:37.868356 27112 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:36:37.870698 27112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786016169045816-21563-0/minicluster-data/master-0/data/instance:
uuid: "e31f3acb9468401f8c938d9979e483f5"
format_stamp: "Formatted at 2026-08-06 11:36:37 on dist-test-slave-7c35"
server_key: "e5fd77697c5489312698c1b432dd81d6"
server_key_iv: "a73e4a197f74dd4a0a5521bb865d57bb"
server_key_version: "encryptionkey@0"
I20260806 11:36:37.871417 27112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786016169045816-21563-0/minicluster-data/master-0/wal/instance:
uuid: "e31f3acb9468401f8c938d9979e483f5"
format_stamp: "Formatted at 2026-08-06 11:36:37 on dist-test-slave-7c35"
server_key: "e5fd77697c5489312698c1b432dd81d6"
server_key_iv: "a73e4a197f74dd4a0a5521bb865d57bb"
server_key_version: "encryptionkey@0"
I20260806 11:36:37.875515 27112 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 11:36:37.878201 27127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:37.879613 27112 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 11:36:37.879771 27112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786016169045816-21563-0/minicluster-data/master-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786016169045816-21563-0/minicluster-data/master-0/wal
uuid: "e31f3acb9468401f8c938d9979e483f5"
format_stamp: "Formatted at 2026-08-06 11:36:37 on dist-test-slave-7c35"
server_key: "e5fd77697c5489312698c1b432dd81d6"
server_key_iv: "a73e4a197f74dd4a0a5521bb865d57bb"
server_key_version: "encryptionkey@0"
I20260806 11:36:37.879900 27112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786016169045816-21563-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786016169045816-21563-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786016169045816-21563-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:36:37.910689 27112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:36:37.911586 27112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:36:37.911818 27112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:36:37.919795 27112 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.254:42739
I20260806 11:36:37.919826 27182 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.254:42739 every 8 connection(s)
I20260806 11:36:37.920871 27112 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
I20260806 11:36:37.922365 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 27112
I20260806 11:36:37.922494 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786016169045816-21563-0/minicluster-data/master-0/wal/instance
I20260806 11:36:37.922804 21563 external_mini_cluster.cc:1428] Setting key cfd75d43567ea31b0cb2eb9e18f7abfc
I20260806 11:36:37.925522 27183 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:37.933972 27183 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e31f3acb9468401f8c938d9979e483f5: Bootstrap starting.
I20260806 11:36:37.935071 27186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:37.924724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32888 (local address 127.21.14.254:42739)
0806 11:36:37.925481 (+ 757us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:37.925517 (+ 36us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:37.925572 (+ 55us) server_negotiation.cc:407] Connection header received
0806 11:36:37.926498 (+ 926us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:37.926526 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:37.927124 (+ 598us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:37.927772 (+ 648us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:37.928272 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:37.929858 (+ 1586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:37.930844 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:37.931615 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:37.931798 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:37.932755 (+ 957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:37.932763 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:37.932783 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 11:36:37.933334 (+ 551us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:37.933417 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 11:36:37.933443 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:37.933551 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:37.933725 (+ 174us) server_negotiation.cc:299] Negotiation successful
0806 11:36:37.934028 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":141,"threads_started":1}
I20260806 11:36:37.936928 27183 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e31f3acb9468401f8c938d9979e483f5: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:37.937898 27183 log.cc:824] T 00000000000000000000000000000000 P e31f3acb9468401f8c938d9979e483f5: Log is configured to *not* fsync() on all Append() calls
I20260806 11:36:37.938818 21563 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 11:36:37.940454 27183 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e31f3acb9468401f8c938d9979e483f5: No bootstrap required, opened a new log
I20260806 11:36:37.944521 27183 raft_consensus.cc:348] T 00000000000000000000000000000000 P e31f3acb9468401f8c938d9979e483f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e31f3acb9468401f8c938d9979e483f5" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 42739 } }
I20260806 11:36:37.944840 27183 raft_consensus.cc:374] T 00000000000000000000000000000000 P e31f3acb9468401f8c938d9979e483f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:37.944942 27183 raft_consensus.cc:729] T 00000000000000000000000000000000 P e31f3acb9468401f8c938d9979e483f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e31f3acb9468401f8c938d9979e483f5, State: Initialized, Role: FOLLOWER
I20260806 11:36:37.945519 27183 consensus_queue.cc:260] T 00000000000000000000000000000000 P e31f3acb9468401f8c938d9979e483f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e31f3acb9468401f8c938d9979e483f5" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 42739 } }
I20260806 11:36:37.945700 27183 raft_consensus.cc:388] T 00000000000000000000000000000000 P e31f3acb9468401f8c938d9979e483f5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 11:36:37.945791 27183 raft_consensus.cc:482] T 00000000000000000000000000000000 P e31f3acb9468401f8c938d9979e483f5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 11:36:37.945930 27183 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e31f3acb9468401f8c938d9979e483f5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:36:37.947093 27183 raft_consensus.cc:504] T 00000000000000000000000000000000 P e31f3acb9468401f8c938d9979e483f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e31f3acb9468401f8c938d9979e483f5" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 42739 } }
I20260806 11:36:37.947470 27183 leader_election.cc:302] T 00000000000000000000000000000000 P e31f3acb9468401f8c938d9979e483f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e31f3acb9468401f8c938d9979e483f5; no voters:
I20260806 11:36:37.947834 27183 leader_election.cc:288] T 00000000000000000000000000000000 P e31f3acb9468401f8c938d9979e483f5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 11:36:37.947985 27188 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e31f3acb9468401f8c938d9979e483f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:36:37.948266 27188 raft_consensus.cc:686] T 00000000000000000000000000000000 P e31f3acb9468401f8c938d9979e483f5 [term 1 LEADER]: Becoming Leader. State: Replica: e31f3acb9468401f8c938d9979e483f5, State: Running, Role: LEADER
I20260806 11:36:37.948609 27188 consensus_queue.cc:237] T 00000000000000000000000000000000 P e31f3acb9468401f8c938d9979e483f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e31f3acb9468401f8c938d9979e483f5" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 42739 } }
I20260806 11:36:37.949416 27183 sys_catalog.cc:564] T 00000000000000000000000000000000 P e31f3acb9468401f8c938d9979e483f5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 11:36:37.950407 27190 sys_catalog.cc:455] T 00000000000000000000000000000000 P e31f3acb9468401f8c938d9979e483f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e31f3acb9468401f8c938d9979e483f5. Latest consensus state: current_term: 1 leader_uuid: "e31f3acb9468401f8c938d9979e483f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e31f3acb9468401f8c938d9979e483f5" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 42739 } } }
I20260806 11:36:37.950567 27190 sys_catalog.cc:458] T 00000000000000000000000000000000 P e31f3acb9468401f8c938d9979e483f5 [sys.catalog]: This master's current role is: LEADER
I20260806 11:36:37.950990 27196 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 11:36:37.951864 27189 sys_catalog.cc:455] T 00000000000000000000000000000000 P e31f3acb9468401f8c938d9979e483f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e31f3acb9468401f8c938d9979e483f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e31f3acb9468401f8c938d9979e483f5" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 42739 } } }
I20260806 11:36:37.951970 27189 sys_catalog.cc:458] T 00000000000000000000000000000000 P e31f3acb9468401f8c938d9979e483f5 [sys.catalog]: This master's current role is: LEADER
I20260806 11:36:37.953972 27196 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 11:36:37.960542 27196 catalog_manager.cc:1347] Generated new cluster ID: ea7e8dd50fb045c583e50aff151a4800
I20260806 11:36:37.960645 27196 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 11:36:37.972882 27196 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 11:36:37.974287 27196 catalog_manager.cc:1504] Loading token signing keys...
I20260806 11:36:37.981895 27196 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e31f3acb9468401f8c938d9979e483f5: Generated new TSK 0
I20260806 11:36:37.983016 27196 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 11:36:38.007843 27215 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260806 11:36:38.007954 27215 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 11:36:38.008802 27186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:37.991331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32900 (local address 127.21.14.254:42739)
0806 11:36:37.991539 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:37.991544 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:37.991772 (+ 228us) server_negotiation.cc:407] Connection header received
0806 11:36:37.991891 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:37.991894 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:37.991972 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:37.992091 (+ 119us) server_negotiation.cc:226] Negotiated authn=JWT
0806 11:36:37.992460 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:37.993011 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:37.994375 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:37.994613 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:37.994708 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:37.995131 (+ 423us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 11:36:38.008328 (+ 13197us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34435/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 11:36:38.008674 (+ 346us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:32900: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34435/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":113}
W20260806 11:36:38.008898 27186 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:32900: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34435/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260806 11:36:38.009032 27214 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:36:37.991179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:42739 (local address 127.0.0.1:32900)
0806 11:36:37.991562 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:37.991581 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:37.991737 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:37.992127 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:37.992131 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:37.992140 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0806 11:36:37.992273 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:37.992280 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:37.993176 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:37.993180 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:37.994221 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:37.994234 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:37.994760 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:37.994765 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:37.994881 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:37.994918 (+ 37us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 11:36:38.008584 (+ 13666us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34435/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 11:36:38.008825 (+ 241us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.21.14.254:42739: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34435/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":120,"threads_started":1}
I20260806 11:36:38.009887 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 27112
2026-08-06T11:36:38Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (353 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260806 11:36:38.053013 21563 mini_oidc.cc:152] Starting JWKS server
I20260806 11:36:38.053912 21563 webserver.cc:460] Webserver started at https://127.0.0.1:32985/ using document root <none> and password file <none>
I20260806 11:36:38.054003 21563 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 11:36:38.054227 21563 webserver.cc:460] Webserver started at http://127.0.0.1:35371/ using document root <none> and password file <none>
2026-08-06T11:36:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T11:36:38Z Disabled control of system clock
I20260806 11:36:38.065232 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786016169045816-21563-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786016169045816-21563-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786016169045816-21563-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:43977
--webserver_interface=127.21.14.254
--webserver_port=0
--builtin_ntp_servers=127.21.14.212:41179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.14.254:43977
--enable_jwt_token_auth=true
--jwks_url=https://localhost:32985/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 11:36:38.201444 27222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:36:38.201872 27222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:36:38.201987 27222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:36:38.206821 27222 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 11:36:38.207006 27222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:36:38.207233 27222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:36:38.207423 27222 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 11:36:38.207731 27222 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 11:36:38.207840 27222 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 11:36:38.207880 27222 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:32985/jwks.json
I20260806 11:36:38.214054 27222 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:41179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786016169045816-21563-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786016169045816-21563-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.14.254:43977
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:43977
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:32985/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.14.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27222
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786016169045816-21563-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:36:38.215457 27222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:36:38.217676 27222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:36:38.228745 27227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:38.228793 27230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:38.228793 27228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:38.228915 27222 server_base.cc:1034] running on GCE node
I20260806 11:36:38.229697 27222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:36:38.231151 27222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:36:38.232400 27222 hybrid_clock.cc:648] HybridClock initialized: now 1786016198232378 us; error 45 us; skew 500 ppm
I20260806 11:36:38.235066 27222 webserver.cc:460] Webserver started at http://127.21.14.254:34717/ using document root <none> and password file <none>
I20260806 11:36:38.235774 27222 fs_manager.cc:359] Metadata directory not provided
I20260806 11:36:38.235885 27222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:36:38.236150 27222 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:36:38.239172 27222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786016169045816-21563-0/minicluster-data/master-0/data/instance:
uuid: "06486db4353642c4882167c5a56fe920"
format_stamp: "Formatted at 2026-08-06 11:36:38 on dist-test-slave-7c35"
server_key: "82c47d38fc981b7a95b87dfaf1a6a8ca"
server_key_iv: "d67107c9c14d8247d1dde25b5264a0bc"
server_key_version: "encryptionkey@0"
I20260806 11:36:38.239907 27222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786016169045816-21563-0/minicluster-data/master-0/wal/instance:
uuid: "06486db4353642c4882167c5a56fe920"
format_stamp: "Formatted at 2026-08-06 11:36:38 on dist-test-slave-7c35"
server_key: "82c47d38fc981b7a95b87dfaf1a6a8ca"
server_key_iv: "d67107c9c14d8247d1dde25b5264a0bc"
server_key_version: "encryptionkey@0"
I20260806 11:36:38.244931 27222 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 11:36:38.248119 27237 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:38.249912 27222 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 11:36:38.250144 27222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786016169045816-21563-0/minicluster-data/master-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786016169045816-21563-0/minicluster-data/master-0/wal
uuid: "06486db4353642c4882167c5a56fe920"
format_stamp: "Formatted at 2026-08-06 11:36:38 on dist-test-slave-7c35"
server_key: "82c47d38fc981b7a95b87dfaf1a6a8ca"
server_key_iv: "d67107c9c14d8247d1dde25b5264a0bc"
server_key_version: "encryptionkey@0"
I20260806 11:36:38.250304 27222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786016169045816-21563-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786016169045816-21563-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786016169045816-21563-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:36:38.283870 27222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:36:38.284745 27222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:36:38.285218 27222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:36:38.293406 27222 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.254:43977
I20260806 11:36:38.293498 27290 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.254:43977 every 8 connection(s)
I20260806 11:36:38.294557 27222 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
I20260806 11:36:38.298439 27291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:38.304339 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 27222
I20260806 11:36:38.304492 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786016169045816-21563-0/minicluster-data/master-0/wal/instance
I20260806 11:36:38.304792 21563 external_mini_cluster.cc:1428] Setting key a8ee5712d6b23150bf9257d0db8c82e0
I20260806 11:36:38.305380 27291 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06486db4353642c4882167c5a56fe920: Bootstrap starting.
I20260806 11:36:38.308430 27291 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 06486db4353642c4882167c5a56fe920: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:38.309382 27291 log.cc:824] T 00000000000000000000000000000000 P 06486db4353642c4882167c5a56fe920: Log is configured to *not* fsync() on all Append() calls
I20260806 11:36:38.311806 27291 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06486db4353642c4882167c5a56fe920: No bootstrap required, opened a new log
I20260806 11:36:38.316025 27294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:38.306439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34942 (local address 127.21.14.254:43977)
0806 11:36:38.307278 (+ 839us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:38.307296 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:38.307336 (+ 40us) server_negotiation.cc:407] Connection header received
0806 11:36:38.308233 (+ 897us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:38.308258 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:38.308810 (+ 552us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:38.309215 (+ 405us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:38.309714 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:38.310772 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:38.311791 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:38.312407 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:38.312600 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:38.313754 (+ 1154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:38.313760 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:38.313778 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 11:36:38.314248 (+ 470us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:38.314315 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 11:36:38.314337 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:38.314463 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:38.314771 (+ 308us) server_negotiation.cc:299] Negotiation successful
0806 11:36:38.315142 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":173,"threads_started":1}
I20260806 11:36:38.316232 27291 raft_consensus.cc:348] T 00000000000000000000000000000000 P 06486db4353642c4882167c5a56fe920 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06486db4353642c4882167c5a56fe920" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 43977 } }
I20260806 11:36:38.316594 27291 raft_consensus.cc:374] T 00000000000000000000000000000000 P 06486db4353642c4882167c5a56fe920 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:38.316653 27291 raft_consensus.cc:729] T 00000000000000000000000000000000 P 06486db4353642c4882167c5a56fe920 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06486db4353642c4882167c5a56fe920, State: Initialized, Role: FOLLOWER
I20260806 11:36:38.317181 27291 consensus_queue.cc:260] T 00000000000000000000000000000000 P 06486db4353642c4882167c5a56fe920 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06486db4353642c4882167c5a56fe920" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 43977 } }
I20260806 11:36:38.317376 27291 raft_consensus.cc:388] T 00000000000000000000000000000000 P 06486db4353642c4882167c5a56fe920 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 11:36:38.317494 27291 raft_consensus.cc:482] T 00000000000000000000000000000000 P 06486db4353642c4882167c5a56fe920 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 11:36:38.317632 27291 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 06486db4353642c4882167c5a56fe920 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:36:38.319203 27291 raft_consensus.cc:504] T 00000000000000000000000000000000 P 06486db4353642c4882167c5a56fe920 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06486db4353642c4882167c5a56fe920" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 43977 } }
I20260806 11:36:38.319649 27291 leader_election.cc:302] T 00000000000000000000000000000000 P 06486db4353642c4882167c5a56fe920 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 06486db4353642c4882167c5a56fe920; no voters:
I20260806 11:36:38.320005 21563 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 11:36:38.320081 27291 leader_election.cc:288] T 00000000000000000000000000000000 P 06486db4353642c4882167c5a56fe920 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 11:36:38.320219 27296 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 06486db4353642c4882167c5a56fe920 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:36:38.320439 27296 raft_consensus.cc:686] T 00000000000000000000000000000000 P 06486db4353642c4882167c5a56fe920 [term 1 LEADER]: Becoming Leader. State: Replica: 06486db4353642c4882167c5a56fe920, State: Running, Role: LEADER
I20260806 11:36:38.320808 27296 consensus_queue.cc:237] T 00000000000000000000000000000000 P 06486db4353642c4882167c5a56fe920 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06486db4353642c4882167c5a56fe920" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 43977 } }
I20260806 11:36:38.321704 27291 sys_catalog.cc:564] T 00000000000000000000000000000000 P 06486db4353642c4882167c5a56fe920 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 11:36:38.322824 27298 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06486db4353642c4882167c5a56fe920 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 06486db4353642c4882167c5a56fe920. Latest consensus state: current_term: 1 leader_uuid: "06486db4353642c4882167c5a56fe920" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06486db4353642c4882167c5a56fe920" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 43977 } } }
I20260806 11:36:38.322913 27297 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06486db4353642c4882167c5a56fe920 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "06486db4353642c4882167c5a56fe920" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06486db4353642c4882167c5a56fe920" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 43977 } } }
I20260806 11:36:38.323020 27298 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06486db4353642c4882167c5a56fe920 [sys.catalog]: This master's current role is: LEADER
I20260806 11:36:38.323050 27297 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06486db4353642c4882167c5a56fe920 [sys.catalog]: This master's current role is: LEADER
I20260806 11:36:38.323437 27306 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 11:36:38.327288 27306 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 11:36:38.334573 27306 catalog_manager.cc:1347] Generated new cluster ID: 0dd891380ac84b878762f4b54db61143
I20260806 11:36:38.334692 27306 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 11:36:38.346145 27306 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 11:36:38.347316 27306 catalog_manager.cc:1504] Loading token signing keys...
I20260806 11:36:38.360991 27306 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 06486db4353642c4882167c5a56fe920: Generated new TSK 0
I20260806 11:36:38.361829 27306 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 11:36:38.376106 27294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:38.370776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34944 (local address 127.21.14.254:43977)
0806 11:36:38.370948 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:38.370955 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:38.371154 (+ 199us) server_negotiation.cc:407] Connection header received
0806 11:36:38.371373 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:38.371376 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:38.371435 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:38.371562 (+ 127us) server_negotiation.cc:226] Negotiated authn=JWT
0806 11:36:38.371824 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:38.372389 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:38.373326 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:38.373569 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:38.373684 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:38.374070 (+ 386us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 11:36:38.375683 (+ 1613us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32985/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 11:36:38.375949 (+ 266us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34944: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32985/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":48}
W20260806 11:36:38.376185 27294 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34944: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32985/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260806 11:36:38.378078 27321 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:36:38.370676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:43977 (local address 127.0.0.1:34944)
0806 11:36:38.371058 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:38.371071 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:38.371211 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:38.371560 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:38.371563 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:38.371572 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0806 11:36:38.371697 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:38.371706 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:38.372524 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:38.372528 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:38.373188 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:38.373197 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:38.373716 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:38.373720 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:38.373818 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:38.373845 (+ 27us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 11:36:38.375954 (+ 2109us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32985/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 11:36:38.377860 (+ 1906us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.21.14.254:43977: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32985/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":291,"mutex_wait_us":1628,"thread_start_us":114,"threads_started":1}
I20260806 11:36:38.378558 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 27222
E20260806 11:36:38.387228 27322 webserver.cc:533] Webserver: sslize failed:
E20260806 11:36:38.387349 27322 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-06T11:36:38Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (370 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-06T11:36:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T11:36:38Z Disabled control of system clock
I20260806 11:36:38.434183 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786016169045816-21563-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786016169045816-21563-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786016169045816-21563-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:36213
--webserver_interface=127.21.14.254
--webserver_port=0
--builtin_ntp_servers=127.21.14.212:46453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.14.254:36213
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786016169045816-21563-0/testchainca.cert with env {}
W20260806 11:36:38.562618 27326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:36:38.563050 27326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:36:38.563107 27326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:36:38.567456 27326 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 11:36:38.567536 27326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:36:38.567552 27326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:36:38.567577 27326 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 11:36:38.567593 27326 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 11:36:38.567610 27326 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 11:36:38.567624 27326 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260806 11:36:38.573524 27326 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:46453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786016169045816-21563-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786016169045816-21563-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.14.254:36213
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:36213
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.14.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786016169045816-21563-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27326
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786016169045816-21563-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:36:38.574712 27326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:36:38.575875 27326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:36:38.585448 27332 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:38.585523 27335 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:38.585520 27333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:38.585983 27326 server_base.cc:1034] running on GCE node
I20260806 11:36:38.586694 27326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:36:38.588109 27326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:36:38.589322 27326 hybrid_clock.cc:648] HybridClock initialized: now 1786016198589294 us; error 49 us; skew 500 ppm
I20260806 11:36:38.591717 27326 webserver.cc:460] Webserver started at http://127.21.14.254:37749/ using document root <none> and password file <none>
I20260806 11:36:38.592346 27326 fs_manager.cc:359] Metadata directory not provided
I20260806 11:36:38.592424 27326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:36:38.592608 27326 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:36:38.595144 27326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786016169045816-21563-0/minicluster-data/master-0/data/instance:
uuid: "dc27180b00b1475abf22d77036474cd5"
format_stamp: "Formatted at 2026-08-06 11:36:38 on dist-test-slave-7c35"
server_key: "684c6d315fa54b0a16af7e9562be0c89"
server_key_iv: "0c696ebbfdfe0bfb1f5a3ce39ab484f4"
server_key_version: "encryptionkey@0"
I20260806 11:36:38.595765 27326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786016169045816-21563-0/minicluster-data/master-0/wal/instance:
uuid: "dc27180b00b1475abf22d77036474cd5"
format_stamp: "Formatted at 2026-08-06 11:36:38 on dist-test-slave-7c35"
server_key: "684c6d315fa54b0a16af7e9562be0c89"
server_key_iv: "0c696ebbfdfe0bfb1f5a3ce39ab484f4"
server_key_version: "encryptionkey@0"
I20260806 11:36:38.600198 27326 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 11:36:38.603426 27342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:38.605086 27326 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 11:36:38.605281 27326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786016169045816-21563-0/minicluster-data/master-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786016169045816-21563-0/minicluster-data/master-0/wal
uuid: "dc27180b00b1475abf22d77036474cd5"
format_stamp: "Formatted at 2026-08-06 11:36:38 on dist-test-slave-7c35"
server_key: "684c6d315fa54b0a16af7e9562be0c89"
server_key_iv: "0c696ebbfdfe0bfb1f5a3ce39ab484f4"
server_key_version: "encryptionkey@0"
I20260806 11:36:38.605443 27326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786016169045816-21563-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786016169045816-21563-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786016169045816-21563-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:36:38.629194 27326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:36:38.630131 27326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:36:38.630375 27326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:36:38.639025 27326 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.254:36213
I20260806 11:36:38.639086 27398 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.254:36213 every 8 connection(s)
I20260806 11:36:38.640205 27326 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
I20260806 11:36:38.642884 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 27326
I20260806 11:36:38.643042 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786016169045816-21563-0/minicluster-data/master-0/wal/instance
I20260806 11:36:38.643374 21563 external_mini_cluster.cc:1428] Setting key 4266471b758f61203c8554bf489426a3
I20260806 11:36:38.643733 27399 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:38.651074 27399 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc27180b00b1475abf22d77036474cd5: Bootstrap starting.
I20260806 11:36:38.653800 27399 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dc27180b00b1475abf22d77036474cd5: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:38.653918 27402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:38.644866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40632 (local address 127.21.14.254:36213)
0806 11:36:38.645641 (+ 775us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:38.645658 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:38.645699 (+ 41us) server_negotiation.cc:407] Connection header received
0806 11:36:38.646888 (+ 1189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:38.646936 (+ 48us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:38.647403 (+ 467us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:38.647774 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:38.648249 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:38.649261 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:38.650209 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:38.650717 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:38.650934 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:38.651984 (+ 1050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:38.651995 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:38.652011 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 11:36:38.652455 (+ 444us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:38.652519 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 11:36:38.652542 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:38.652683 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:38.652824 (+ 141us) server_negotiation.cc:299] Negotiation successful
0806 11:36:38.653122 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":168,"threads_started":1}
I20260806 11:36:38.654865 27399 log.cc:824] T 00000000000000000000000000000000 P dc27180b00b1475abf22d77036474cd5: Log is configured to *not* fsync() on all Append() calls
I20260806 11:36:38.657416 27399 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc27180b00b1475abf22d77036474cd5: No bootstrap required, opened a new log
I20260806 11:36:38.657619 21563 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 11:36:38.661780 27399 raft_consensus.cc:348] T 00000000000000000000000000000000 P dc27180b00b1475abf22d77036474cd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc27180b00b1475abf22d77036474cd5" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 36213 } }
I20260806 11:36:38.662060 27399 raft_consensus.cc:374] T 00000000000000000000000000000000 P dc27180b00b1475abf22d77036474cd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:38.662099 27399 raft_consensus.cc:729] T 00000000000000000000000000000000 P dc27180b00b1475abf22d77036474cd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc27180b00b1475abf22d77036474cd5, State: Initialized, Role: FOLLOWER
I20260806 11:36:38.662499 27399 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc27180b00b1475abf22d77036474cd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc27180b00b1475abf22d77036474cd5" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 36213 } }
I20260806 11:36:38.662629 27399 raft_consensus.cc:388] T 00000000000000000000000000000000 P dc27180b00b1475abf22d77036474cd5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 11:36:38.662673 27399 raft_consensus.cc:482] T 00000000000000000000000000000000 P dc27180b00b1475abf22d77036474cd5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 11:36:38.662751 27399 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dc27180b00b1475abf22d77036474cd5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:36:38.663848 27399 raft_consensus.cc:504] T 00000000000000000000000000000000 P dc27180b00b1475abf22d77036474cd5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc27180b00b1475abf22d77036474cd5" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 36213 } }
I20260806 11:36:38.664266 27399 leader_election.cc:302] T 00000000000000000000000000000000 P dc27180b00b1475abf22d77036474cd5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dc27180b00b1475abf22d77036474cd5; no voters:
I20260806 11:36:38.664600 27399 leader_election.cc:288] T 00000000000000000000000000000000 P dc27180b00b1475abf22d77036474cd5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 11:36:38.664799 27404 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dc27180b00b1475abf22d77036474cd5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:36:38.665069 27404 raft_consensus.cc:686] T 00000000000000000000000000000000 P dc27180b00b1475abf22d77036474cd5 [term 1 LEADER]: Becoming Leader. State: Replica: dc27180b00b1475abf22d77036474cd5, State: Running, Role: LEADER
I20260806 11:36:38.665441 27404 consensus_queue.cc:237] T 00000000000000000000000000000000 P dc27180b00b1475abf22d77036474cd5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc27180b00b1475abf22d77036474cd5" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 36213 } }
I20260806 11:36:38.666141 27399 sys_catalog.cc:564] T 00000000000000000000000000000000 P dc27180b00b1475abf22d77036474cd5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 11:36:38.667587 27405 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc27180b00b1475abf22d77036474cd5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dc27180b00b1475abf22d77036474cd5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc27180b00b1475abf22d77036474cd5" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 36213 } } }
I20260806 11:36:38.667587 27406 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc27180b00b1475abf22d77036474cd5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc27180b00b1475abf22d77036474cd5. Latest consensus state: current_term: 1 leader_uuid: "dc27180b00b1475abf22d77036474cd5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc27180b00b1475abf22d77036474cd5" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 36213 } } }
I20260806 11:36:38.667788 27405 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc27180b00b1475abf22d77036474cd5 [sys.catalog]: This master's current role is: LEADER
I20260806 11:36:38.667788 27406 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc27180b00b1475abf22d77036474cd5 [sys.catalog]: This master's current role is: LEADER
I20260806 11:36:38.668237 27414 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 11:36:38.672056 27414 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 11:36:38.680326 27414 catalog_manager.cc:1347] Generated new cluster ID: a13c6b62b0d54e9a831eb18e98396dc1
I20260806 11:36:38.680434 27414 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 11:36:38.688489 27414 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 11:36:38.689532 27414 catalog_manager.cc:1504] Loading token signing keys...
I20260806 11:36:38.696887 27414 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dc27180b00b1475abf22d77036474cd5: Generated new TSK 0
I20260806 11:36:38.697726 27414 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 11:36:39.168488 27402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:38.713312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40640 (local address 127.21.14.254:36213)
0806 11:36:38.713505 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:38.713510 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:38.713705 (+ 195us) server_negotiation.cc:407] Connection header received
0806 11:36:38.713895 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:38.713899 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:38.713957 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:38.714061 (+ 104us) server_negotiation.cc:226] Negotiated authn=JWT
0806 11:36:38.714402 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:38.715049 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:38.716319 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:38.716546 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:38.716634 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:38.717024 (+ 390us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 11:36:39.168048 (+451024us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 11:36:39.168355 (+ 307us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40640: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":92}
W20260806 11:36:39.168598 27402 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40640: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260806 11:36:39.168957 27429 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:36:38.713235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:36213 (local address 127.0.0.1:40640)
0806 11:36:38.713594 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:38.713610 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:38.713751 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:38.714094 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:38.714097 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:38.714105 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0806 11:36:38.714231 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:38.714237 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:38.715169 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:38.715173 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:38.716183 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:38.716195 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:38.716687 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:38.716691 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:38.716793 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:38.716826 (+ 33us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 11:36:39.168331 (+451505us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 11:36:39.168709 (+ 378us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.21.14.254:36213: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":152,"threads_started":1}
I20260806 11:36:39.169708 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 27326
2026-08-06T11:36:39Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (769 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (149 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (141 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 11:36:39 dist-test-slave-7c35 krb5kdc[27442](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 11:36:39 dist-test-slave-7c35 krb5kdc[27442](info): set up 2 sockets
Aug 06 11:36:39 dist-test-slave-7c35 krb5kdc[27442](info): commencing operation
krb5kdc: starting...
W20260806 11:36:40.567373 21563 mini_kdc.cc:121] Time spent starting KDC: real 1.083s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Aug 06 11:36:40 dist-test-slave-7c35 krb5kdc[27442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016200, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T11:36:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T11:36:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.14.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab.
Entry for principal kudu/127.21.14.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab.
I20260806 11:36:40.717558 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:34529
--webserver_interface=127.21.14.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab
--principal=kudu/127.21.14.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:45009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.14.254:34529
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:36:40.842443 27455 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:36:40.842839 27455 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:36:40.842952 27455 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:36:40.847381 27455 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 11:36:40.847512 27455 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:36:40.847548 27455 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:36:40.847579 27455 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 11:36:40.847637 27455 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 11:36:40.853534 27455 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:45009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.14.254:34529
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab
--principal=kudu/127.21.14.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:34529
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.14.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27455
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:36:40.854832 27455 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:36:40.856046 27455 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:36:40.866526 27463 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:40.866540 27460 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:40.866608 27455 server_base.cc:1034] running on GCE node
W20260806 11:36:40.866540 27461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:40.867584 27455 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:36:40.868935 27455 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:36:40.870155 27455 hybrid_clock.cc:648] HybridClock initialized: now 1786016200870109 us; error 52 us; skew 500 ppm
Aug 06 11:36:40 dist-test-slave-7c35 krb5kdc[27442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016200, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:36:40.881197 27455 init.cc:377] Logged in from keytab as kudu/127.21.14.254@KRBTEST.COM (short username kudu)
I20260806 11:36:40.882414 27455 webserver.cc:460] Webserver started at http://127.21.14.254:34353/ using document root <none> and password file <none>
I20260806 11:36:40.883176 27455 fs_manager.cc:359] Metadata directory not provided
I20260806 11:36:40.883258 27455 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:36:40.883522 27455 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:36:40.885887 27455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/minicluster-data/master-0/data/instance:
uuid: "37319a47a4e44fc0889515db14eeb1b0"
format_stamp: "Formatted at 2026-08-06 11:36:40 on dist-test-slave-7c35"
server_key: "591f0b054adf81d5a4d0238fd0224ff9"
server_key_iv: "f2f3013a34366bf799c48a019bf1b748"
server_key_version: "encryptionkey@0"
I20260806 11:36:40.886574 27455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/minicluster-data/master-0/wal/instance:
uuid: "37319a47a4e44fc0889515db14eeb1b0"
format_stamp: "Formatted at 2026-08-06 11:36:40 on dist-test-slave-7c35"
server_key: "591f0b054adf81d5a4d0238fd0224ff9"
server_key_iv: "f2f3013a34366bf799c48a019bf1b748"
server_key_version: "encryptionkey@0"
I20260806 11:36:40.891201 27455 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 11:36:40.894241 27470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:40.895897 27455 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 11:36:40.896101 27455 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/minicluster-data/master-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/minicluster-data/master-0/wal
uuid: "37319a47a4e44fc0889515db14eeb1b0"
format_stamp: "Formatted at 2026-08-06 11:36:40 on dist-test-slave-7c35"
server_key: "591f0b054adf81d5a4d0238fd0224ff9"
server_key_iv: "f2f3013a34366bf799c48a019bf1b748"
server_key_version: "encryptionkey@0"
I20260806 11:36:40.896253 27455 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:36:40.912693 27455 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:36:40.917279 27455 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:36:40.917634 27455 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:36:40.925485 27455 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.254:34529
I20260806 11:36:40.925565 27526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.254:34529 every 8 connection(s)
I20260806 11:36:40.926611 27455 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
I20260806 11:36:40.927198 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 27455
I20260806 11:36:40.927358 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/minicluster-data/master-0/wal/instance
I20260806 11:36:40.927690 21563 external_mini_cluster.cc:1428] Setting key 7335212f60f5abff8efa09a5fa0865d3
I20260806 11:36:40.930037 27527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:40.937587 27527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37319a47a4e44fc0889515db14eeb1b0: Bootstrap starting.
Aug 06 11:36:40 dist-test-slave-7c35 krb5kdc[27442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016200, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:36:40.940289 27527 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 37319a47a4e44fc0889515db14eeb1b0: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:40.941133 27527 log.cc:824] T 00000000000000000000000000000000 P 37319a47a4e44fc0889515db14eeb1b0: Log is configured to *not* fsync() on all Append() calls
I20260806 11:36:40.943607 27527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37319a47a4e44fc0889515db14eeb1b0: No bootstrap required, opened a new log
I20260806 11:36:40.944245 27530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:40.929432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36898 (local address 127.21.14.254:34529)
0806 11:36:40.929881 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:40.929898 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:40.929936 (+ 38us) server_negotiation.cc:407] Connection header received
0806 11:36:40.930655 (+ 719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:40.930675 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:40.931100 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:40.931405 (+ 305us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:40.932430 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:40.933539 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:40.934740 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:40.935355 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:40.935500 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:40.938570 (+ 3070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:40.938576 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:40.938591 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:40.938625 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:40.940807 (+ 2182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:40.941531 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:40.941545 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:40.941550 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:40.941684 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:40.942011 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:40.942015 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:40.942016 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:40.942479 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:40.942695 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:40.943067 (+ 372us) server_negotiation.cc:299] Negotiation successful
0806 11:36:40.943388 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":155,"threads_started":1}
I20260806 11:36:40.947708 27527 raft_consensus.cc:348] T 00000000000000000000000000000000 P 37319a47a4e44fc0889515db14eeb1b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37319a47a4e44fc0889515db14eeb1b0" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 34529 } }
I20260806 11:36:40.947997 27527 raft_consensus.cc:374] T 00000000000000000000000000000000 P 37319a47a4e44fc0889515db14eeb1b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:40.948084 27527 raft_consensus.cc:729] T 00000000000000000000000000000000 P 37319a47a4e44fc0889515db14eeb1b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37319a47a4e44fc0889515db14eeb1b0, State: Initialized, Role: FOLLOWER
I20260806 11:36:40.948546 27527 consensus_queue.cc:260] T 00000000000000000000000000000000 P 37319a47a4e44fc0889515db14eeb1b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37319a47a4e44fc0889515db14eeb1b0" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 34529 } }
I20260806 11:36:40.948704 27527 raft_consensus.cc:388] T 00000000000000000000000000000000 P 37319a47a4e44fc0889515db14eeb1b0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 11:36:40.948810 27527 raft_consensus.cc:482] T 00000000000000000000000000000000 P 37319a47a4e44fc0889515db14eeb1b0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 11:36:40.948896 27527 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 37319a47a4e44fc0889515db14eeb1b0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:36:40.949965 27527 raft_consensus.cc:504] T 00000000000000000000000000000000 P 37319a47a4e44fc0889515db14eeb1b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37319a47a4e44fc0889515db14eeb1b0" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 34529 } }
I20260806 11:36:40.950376 27527 leader_election.cc:302] T 00000000000000000000000000000000 P 37319a47a4e44fc0889515db14eeb1b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 37319a47a4e44fc0889515db14eeb1b0; no voters:
I20260806 11:36:40.950783 27527 leader_election.cc:288] T 00000000000000000000000000000000 P 37319a47a4e44fc0889515db14eeb1b0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 11:36:40.950935 27532 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 37319a47a4e44fc0889515db14eeb1b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:36:40.951210 27532 raft_consensus.cc:686] T 00000000000000000000000000000000 P 37319a47a4e44fc0889515db14eeb1b0 [term 1 LEADER]: Becoming Leader. State: Replica: 37319a47a4e44fc0889515db14eeb1b0, State: Running, Role: LEADER
I20260806 11:36:40.951602 27532 consensus_queue.cc:237] T 00000000000000000000000000000000 P 37319a47a4e44fc0889515db14eeb1b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37319a47a4e44fc0889515db14eeb1b0" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 34529 } }
I20260806 11:36:40.952239 27527 sys_catalog.cc:564] T 00000000000000000000000000000000 P 37319a47a4e44fc0889515db14eeb1b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 11:36:40.953402 27534 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37319a47a4e44fc0889515db14eeb1b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 37319a47a4e44fc0889515db14eeb1b0. Latest consensus state: current_term: 1 leader_uuid: "37319a47a4e44fc0889515db14eeb1b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37319a47a4e44fc0889515db14eeb1b0" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 34529 } } }
I20260806 11:36:40.953547 27534 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37319a47a4e44fc0889515db14eeb1b0 [sys.catalog]: This master's current role is: LEADER
I20260806 11:36:40.953969 27541 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 11:36:40.953930 27533 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37319a47a4e44fc0889515db14eeb1b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "37319a47a4e44fc0889515db14eeb1b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37319a47a4e44fc0889515db14eeb1b0" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 34529 } } }
I20260806 11:36:40.954104 27533 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37319a47a4e44fc0889515db14eeb1b0 [sys.catalog]: This master's current role is: LEADER
I20260806 11:36:40.957916 27541 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 11:36:40.964666 27541 catalog_manager.cc:1347] Generated new cluster ID: e73b06e60f8a4bdba9cfcdd84647d663
I20260806 11:36:40.964762 27541 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 11:36:40.973716 27541 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 11:36:40.974694 27541 catalog_manager.cc:1504] Loading token signing keys...
I20260806 11:36:40.982007 27541 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 37319a47a4e44fc0889515db14eeb1b0: Generated new TSK 0
I20260806 11:36:40.982872 27541 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.14.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab.
Entry for principal kudu/127.21.14.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab.
I20260806 11:36:41.019017 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.193:0
--local_ip_for_outbound_sockets=127.21.14.193
--webserver_interface=127.21.14.193
--webserver_port=0
--tserver_master_addrs=127.21.14.254:34529
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab
--principal=kudu/127.21.14.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:45009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:36:41.148324 27554 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:36:41.148679 27554 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:36:41.148728 27554 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:36:41.153216 27554 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:36:41.153306 27554 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:36:41.153371 27554 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.193
I20260806 11:36:41.159489 27554 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:45009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab
--principal=kudu/127.21.14.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.14.193
--webserver_port=0
--tserver_master_addrs=127.21.14.254:34529
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27554
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.193
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:36:41.160682 27554 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:36:41.161774 27554 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:36:41.173105 27562 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:41.173105 27560 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:41.173296 27554 server_base.cc:1034] running on GCE node
W20260806 11:36:41.173482 27559 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:41.174010 27554 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:36:41.174733 27554 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:36:41.175936 27554 hybrid_clock.cc:648] HybridClock initialized: now 1786016201175914 us; error 33 us; skew 500 ppm
Aug 06 11:36:41 dist-test-slave-7c35 krb5kdc[27442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016201, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:36:41.179368 27554 init.cc:377] Logged in from keytab as kudu/127.21.14.193@KRBTEST.COM (short username kudu)
I20260806 11:36:41.180450 27554 webserver.cc:460] Webserver started at http://127.21.14.193:36929/ using document root <none> and password file <none>
I20260806 11:36:41.181098 27554 fs_manager.cc:359] Metadata directory not provided
I20260806 11:36:41.181175 27554 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:36:41.181452 27554 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:36:41.183995 27554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/minicluster-data/ts-0/data/instance:
uuid: "0c7c785978594b36880ec59900f72a6e"
format_stamp: "Formatted at 2026-08-06 11:36:41 on dist-test-slave-7c35"
server_key: "abed75bcf9045d960b1423bdc5c676d3"
server_key_iv: "4a81ee2b1b846eaf0f623768afcf843b"
server_key_version: "encryptionkey@0"
I20260806 11:36:41.184719 27554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/minicluster-data/ts-0/wal/instance:
uuid: "0c7c785978594b36880ec59900f72a6e"
format_stamp: "Formatted at 2026-08-06 11:36:41 on dist-test-slave-7c35"
server_key: "abed75bcf9045d960b1423bdc5c676d3"
server_key_iv: "4a81ee2b1b846eaf0f623768afcf843b"
server_key_version: "encryptionkey@0"
I20260806 11:36:41.190130 27554 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 11:36:41.193284 27569 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:41.194773 27554 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 11:36:41.195007 27554 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/minicluster-data/ts-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/minicluster-data/ts-0/wal
uuid: "0c7c785978594b36880ec59900f72a6e"
format_stamp: "Formatted at 2026-08-06 11:36:41 on dist-test-slave-7c35"
server_key: "abed75bcf9045d960b1423bdc5c676d3"
server_key_iv: "4a81ee2b1b846eaf0f623768afcf843b"
server_key_version: "encryptionkey@0"
I20260806 11:36:41.195158 27554 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:36:41.214484 27554 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:36:41.218470 27554 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:36:41.218734 27554 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:36:41.219484 27554 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:36:41.220419 27554 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:36:41.220467 27554 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:41.220541 27554 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:36:41.220573 27554 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:41.230827 27554 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.193:44191
I20260806 11:36:41.231066 27688 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.193:44191 every 8 connection(s)
I20260806 11:36:41.231935 27554 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
I20260806 11:36:41.237835 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 27554
I20260806 11:36:41.238042 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/minicluster-data/ts-0/wal/instance
I20260806 11:36:41.238353 21563 external_mini_cluster.cc:1428] Setting key 81c75f96d32e77bc213e0997efec5cf9
Aug 06 11:36:41 dist-test-slave-7c35 krb5kdc[27442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016201, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.193@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:36:41.250488 27530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:41.234085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:60361 (local address 127.21.14.254:34529)
0806 11:36:41.234275 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:41.234281 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:41.235266 (+ 985us) server_negotiation.cc:407] Connection header received
0806 11:36:41.236736 (+ 1470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:41.236739 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:41.236796 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:41.236892 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:41.238587 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:41.239367 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:41.241176 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:41.241599 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:41.241749 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:41.245394 (+ 3645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:41.245397 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:41.245400 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:41.245433 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:41.247542 (+ 2109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:41.248463 (+ 921us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:41.248466 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:41.248467 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:41.248586 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:41.249089 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:41.249091 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:41.249092 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:41.249311 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:41.249396 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:41.250218 (+ 822us) server_negotiation.cc:299] Negotiation successful
0806 11:36:41.250351 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
WARNING: no policy specified for kudu/127.21.14.194@KRBTEST.COM; defaulting to no policy
I20260806 11:36:41.252014 27691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:41.234496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:34529 (local address 127.21.14.193:60361)
0806 11:36:41.235090 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:41.235141 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:41.236444 (+ 1303us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:41.237159 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:41.237167 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:41.237603 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:41.238313 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:41.238335 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:41.239659 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:41.239664 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:41.241001 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:41.241015 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:41.241962 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:41.241966 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:41.242154 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:41.243113 (+ 959us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:41.243134 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:41.244991 (+ 1857us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:41.247733 (+ 2742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:41.247745 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:41.247776 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:41.248311 (+ 535us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:41.248715 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:41.248722 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:41.248725 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:41.248972 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:41.249435 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:41.249444 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:41.249891 (+ 447us) client_negotiation.cc:769] Sending connection context
0806 11:36:41.250184 (+ 293us) client_negotiation.cc:240] Negotiation successful
0806 11:36:41.250615 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":130,"threads_started":1}
I20260806 11:36:41.253532 27689 heartbeater.cc:347] Connected to a master server at 127.21.14.254:34529
I20260806 11:36:41.253829 27689 heartbeater.cc:464] Registering TS with master...
I20260806 11:36:41.254630 27689 heartbeater.cc:511] Master 127.21.14.254:34529 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.194@KRBTEST.COM" created.
I20260806 11:36:41.256500 27488 ts_manager.cc:194] Registered new tserver with Master: 0c7c785978594b36880ec59900f72a6e (127.21.14.193:44191)
I20260806 11:36:41.257792 27488 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.14.193@KRBTEST.COM'} at 127.21.14.193:60361
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab.
Entry for principal kudu/127.21.14.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab.
I20260806 11:36:41.275120 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.194:0
--local_ip_for_outbound_sockets=127.21.14.194
--webserver_interface=127.21.14.194
--webserver_port=0
--tserver_master_addrs=127.21.14.254:34529
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab
--principal=kudu/127.21.14.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:45009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:36:41.395856 27694 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:36:41.396150 27694 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:36:41.396195 27694 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:36:41.400588 27694 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:36:41.400662 27694 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:36:41.400725 27694 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.194
I20260806 11:36:41.406320 27694 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:45009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab
--principal=kudu/127.21.14.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.14.194
--webserver_port=0
--tserver_master_addrs=127.21.14.254:34529
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27694
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.194
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:36:41.407809 27694 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:36:41.408973 27694 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:36:41.419822 27699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:41.419951 27694 server_base.cc:1034] running on GCE node
W20260806 11:36:41.419909 27700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:41.420104 27702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:41.420805 27694 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:36:41.421599 27694 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:36:41.425006 27694 hybrid_clock.cc:648] HybridClock initialized: now 1786016201424896 us; error 106 us; skew 500 ppm
Aug 06 11:36:41 dist-test-slave-7c35 krb5kdc[27442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016201, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:36:41.428644 27694 init.cc:377] Logged in from keytab as kudu/127.21.14.194@KRBTEST.COM (short username kudu)
I20260806 11:36:41.429759 27694 webserver.cc:460] Webserver started at http://127.21.14.194:40961/ using document root <none> and password file <none>
I20260806 11:36:41.430418 27694 fs_manager.cc:359] Metadata directory not provided
I20260806 11:36:41.430523 27694 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:36:41.430788 27694 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:36:41.433148 27694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/minicluster-data/ts-1/data/instance:
uuid: "48ef016dbc6945ff85e8ed843d06b0dd"
format_stamp: "Formatted at 2026-08-06 11:36:41 on dist-test-slave-7c35"
server_key: "c257bd45ef6c37f06c2b3331422f6929"
server_key_iv: "d00c61832600149152488a6cb6dd09c5"
server_key_version: "encryptionkey@0"
I20260806 11:36:41.433818 27694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/minicluster-data/ts-1/wal/instance:
uuid: "48ef016dbc6945ff85e8ed843d06b0dd"
format_stamp: "Formatted at 2026-08-06 11:36:41 on dist-test-slave-7c35"
server_key: "c257bd45ef6c37f06c2b3331422f6929"
server_key_iv: "d00c61832600149152488a6cb6dd09c5"
server_key_version: "encryptionkey@0"
I20260806 11:36:41.438078 27694 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260806 11:36:41.440850 27709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:41.442274 27694 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 11:36:41.442447 27694 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/minicluster-data/ts-1/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/minicluster-data/ts-1/wal
uuid: "48ef016dbc6945ff85e8ed843d06b0dd"
format_stamp: "Formatted at 2026-08-06 11:36:41 on dist-test-slave-7c35"
server_key: "c257bd45ef6c37f06c2b3331422f6929"
server_key_iv: "d00c61832600149152488a6cb6dd09c5"
server_key_version: "encryptionkey@0"
I20260806 11:36:41.442586 27694 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:36:41.465437 27694 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:36:41.468730 27694 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:36:41.468964 27694 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:36:41.469640 27694 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:36:41.470566 27694 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:36:41.470650 27694 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:41.470750 27694 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:36:41.470803 27694 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:41.480532 27694 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.194:43047
I20260806 11:36:41.480556 27826 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.194:43047 every 8 connection(s)
I20260806 11:36:41.481617 27694 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
I20260806 11:36:41.483547 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 27694
I20260806 11:36:41.483661 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/minicluster-data/ts-1/wal/instance
I20260806 11:36:41.483976 21563 external_mini_cluster.cc:1428] Setting key e87d976fc5461dda4601191b68054303
Aug 06 11:36:41 dist-test-slave-7c35 krb5kdc[27442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016201, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.194@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.14.195@KRBTEST.COM; defaulting to no policy
I20260806 11:36:41.498512 27530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:41.483680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:36625 (local address 127.21.14.254:34529)
0806 11:36:41.483825 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:41.483830 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:41.484962 (+ 1132us) server_negotiation.cc:407] Connection header received
0806 11:36:41.486576 (+ 1614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:41.486583 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:41.486647 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:41.486752 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:41.488303 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:41.488846 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:41.490759 (+ 1913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:41.491148 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:41.491243 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:41.494491 (+ 3248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:41.494494 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:41.494497 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:41.494527 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:41.496257 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:41.496872 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:41.496875 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:41.496877 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:41.496965 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:41.497319 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:41.497322 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:41.497323 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:41.497537 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:41.497618 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:41.498229 (+ 611us) server_negotiation.cc:299] Negotiation successful
0806 11:36:41.498346 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.195@KRBTEST.COM" created.
I20260806 11:36:41.499732 27829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:41.484207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:34529 (local address 127.21.14.194:36625)
0806 11:36:41.484761 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:41.484846 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:41.486338 (+ 1492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:41.486984 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:41.486992 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:41.487396 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:41.488098 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:41.488122 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:41.488991 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:41.488998 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:41.490596 (+ 1598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:41.490614 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:41.491337 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:41.491342 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:41.491579 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:41.492282 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:41.492312 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:41.494253 (+ 1941us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:41.496395 (+ 2142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:41.496401 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:41.496418 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:41.496741 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:41.497066 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:41.497069 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:41.497070 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:41.497226 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:41.497658 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:41.497664 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:41.497943 (+ 279us) client_negotiation.cc:769] Sending connection context
0806 11:36:41.498123 (+ 180us) client_negotiation.cc:240] Negotiation successful
0806 11:36:41.498415 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":136,"threads_started":1}
I20260806 11:36:41.501163 27827 heartbeater.cc:347] Connected to a master server at 127.21.14.254:34529
I20260806 11:36:41.501459 27827 heartbeater.cc:464] Registering TS with master...
I20260806 11:36:41.502080 27827 heartbeater.cc:511] Master 127.21.14.254:34529 requested a full tablet report, sending...
I20260806 11:36:41.503386 27488 ts_manager.cc:194] Registered new tserver with Master: 48ef016dbc6945ff85e8ed843d06b0dd (127.21.14.194:43047)
I20260806 11:36:41.504019 27488 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.14.194@KRBTEST.COM'} at 127.21.14.194:36625
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab.
Entry for principal kudu/127.21.14.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab.
I20260806 11:36:41.519340 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.195:0
--local_ip_for_outbound_sockets=127.21.14.195
--webserver_interface=127.21.14.195
--webserver_port=0
--tserver_master_addrs=127.21.14.254:34529
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab
--principal=kudu/127.21.14.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:45009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:36:41.649132 27833 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:36:41.649470 27833 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:36:41.649534 27833 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:36:41.654021 27833 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:36:41.654124 27833 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:36:41.654192 27833 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.195
I20260806 11:36:41.660002 27833 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:45009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab
--principal=kudu/127.21.14.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.14.195
--webserver_port=0
--tserver_master_addrs=127.21.14.254:34529
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27833
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.195
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:36:41.661180 27833 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:36:41.662257 27833 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:36:41.673262 27842 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:41.673337 27840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:41.673262 27839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:41.673759 27833 server_base.cc:1034] running on GCE node
I20260806 11:36:41.674249 27833 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:36:41.675120 27833 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:36:41.676355 27833 hybrid_clock.cc:648] HybridClock initialized: now 1786016201676328 us; error 32 us; skew 500 ppm
Aug 06 11:36:41 dist-test-slave-7c35 krb5kdc[27442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016201, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:36:41.679775 27833 init.cc:377] Logged in from keytab as kudu/127.21.14.195@KRBTEST.COM (short username kudu)
I20260806 11:36:41.680799 27833 webserver.cc:460] Webserver started at http://127.21.14.195:45041/ using document root <none> and password file <none>
I20260806 11:36:41.681465 27833 fs_manager.cc:359] Metadata directory not provided
I20260806 11:36:41.681569 27833 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:36:41.681830 27833 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:36:41.684253 27833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/minicluster-data/ts-2/data/instance:
uuid: "3333d337a7444d66a579bbaa43d07b71"
format_stamp: "Formatted at 2026-08-06 11:36:41 on dist-test-slave-7c35"
server_key: "2c1a24925c8c54be88c703b6765da104"
server_key_iv: "5702724381f0842dd443f7df2f9b80c5"
server_key_version: "encryptionkey@0"
I20260806 11:36:41.684906 27833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/minicluster-data/ts-2/wal/instance:
uuid: "3333d337a7444d66a579bbaa43d07b71"
format_stamp: "Formatted at 2026-08-06 11:36:41 on dist-test-slave-7c35"
server_key: "2c1a24925c8c54be88c703b6765da104"
server_key_iv: "5702724381f0842dd443f7df2f9b80c5"
server_key_version: "encryptionkey@0"
I20260806 11:36:41.689296 27833 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 11:36:41.692118 27849 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:41.693495 27833 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 11:36:41.693648 27833 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/minicluster-data/ts-2/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/minicluster-data/ts-2/wal
uuid: "3333d337a7444d66a579bbaa43d07b71"
format_stamp: "Formatted at 2026-08-06 11:36:41 on dist-test-slave-7c35"
server_key: "2c1a24925c8c54be88c703b6765da104"
server_key_iv: "5702724381f0842dd443f7df2f9b80c5"
server_key_version: "encryptionkey@0"
I20260806 11:36:41.693778 27833 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:36:41.713539 27833 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:36:41.717077 27833 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:36:41.717309 27833 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:36:41.717988 27833 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:36:41.718955 27833 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:36:41.719005 27833 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:41.719049 27833 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:36:41.719074 27833 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:41.728479 27833 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.195:41129
I20260806 11:36:41.728498 27970 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.195:41129 every 8 connection(s)
I20260806 11:36:41.729503 27833 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
I20260806 11:36:41.738013 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 27833
I20260806 11:36:41.738134 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/minicluster-data/ts-2/wal/instance
I20260806 11:36:41.738458 21563 external_mini_cluster.cc:1428] Setting key 06300eb876a67e94a2ed299c5c778b2e
Aug 06 11:36:41 dist-test-slave-7c35 krb5kdc[27442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016201, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.195@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:36:41.745596 27530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:41.731501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:37433 (local address 127.21.14.254:34529)
0806 11:36:41.731666 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:41.731670 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:41.732625 (+ 955us) server_negotiation.cc:407] Connection header received
0806 11:36:41.734215 (+ 1590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:41.734218 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:41.734285 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:41.734405 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:41.735761 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:41.736317 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:41.737561 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:41.737793 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:41.737914 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:41.741332 (+ 3418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:41.741335 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:41.741337 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:41.741365 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:41.743417 (+ 2052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:41.744075 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:41.744078 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:41.744079 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:41.744165 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:41.744507 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:41.744509 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:41.744510 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:41.744718 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:41.744799 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:41.745336 (+ 537us) server_negotiation.cc:299] Negotiation successful
0806 11:36:41.745455 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 11:36:41.746666 27973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:41.731920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:34529 (local address 127.21.14.195:37433)
0806 11:36:41.732432 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:41.732479 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:41.733966 (+ 1487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:41.734619 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:41.734627 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:41.735048 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:41.735586 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:41.735601 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:41.736464 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:41.736474 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:41.737415 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:41.737426 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:41.737973 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:41.737978 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:41.738240 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:41.739127 (+ 887us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:41.739158 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:41.741097 (+ 1939us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:41.743607 (+ 2510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:41.743615 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:41.743633 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:41.743961 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:41.744258 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:41.744260 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:41.744261 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:41.744420 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:41.744812 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:41.744818 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:41.745104 (+ 286us) client_negotiation.cc:769] Sending connection context
0806 11:36:41.745280 (+ 176us) client_negotiation.cc:240] Negotiation successful
0806 11:36:41.745572 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":128,"threads_started":1}
I20260806 11:36:41.748131 27971 heartbeater.cc:347] Connected to a master server at 127.21.14.254:34529
I20260806 11:36:41.748404 27971 heartbeater.cc:464] Registering TS with master...
I20260806 11:36:41.748963 27971 heartbeater.cc:511] Master 127.21.14.254:34529 requested a full tablet report, sending...
I20260806 11:36:41.750170 27488 ts_manager.cc:194] Registered new tserver with Master: 3333d337a7444d66a579bbaa43d07b71 (127.21.14.195:41129)
I20260806 11:36:41.750762 27488 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.14.195@KRBTEST.COM'} at 127.21.14.195:37433
I20260806 11:36:41.753173 21563 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/krb5kdc/test-admin.keytab.
Aug 06 11:36:41 dist-test-slave-7c35 krb5kdc[27442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016201, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:36:41.773964 21563 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260806 11:36:41.777170 27978 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:36:41.775695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54186 (local address 127.21.14.193:44191)
0806 11:36:41.776075 (+ 380us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:41.776081 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:41.776127 (+ 46us) server_negotiation.cc:407] Connection header received
0806 11:36:41.776179 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:41.776184 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:41.776377 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:41.776500 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:41.777030 (+ 530us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54186: BlockingRecv error: recv got EOF from 127.0.0.1:54186 (error 108)
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":129,"threads_started":1}
W20260806 11:36:41.777396 21563 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/krb5kdc/krb5cc' of type FILE
Aug 06 11:36:41 dist-test-slave-7c35 krb5kdc[27442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016201, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:36:41.778644 21563 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 11:36:41 dist-test-slave-7c35 krb5kdc[27442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016201, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:36:41.779706 21563 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 11:36:41 dist-test-slave-7c35 krb5kdc[27442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016201, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.14.193@KRBTEST.COM
I20260806 11:36:41.791607 27978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:41.780782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54188 (local address 127.21.14.193:44191)
0806 11:36:41.780927 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:41.780932 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:41.781041 (+ 109us) server_negotiation.cc:407] Connection header received
0806 11:36:41.781230 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:41.781232 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:41.781279 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:41.781367 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:41.781998 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:41.782806 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:41.783759 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:41.784209 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:41.784335 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:41.787152 (+ 2817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:41.787166 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:41.787188 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:41.787220 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:41.789221 (+ 2001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:41.789753 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:41.789761 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:41.789762 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:41.789864 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:41.790340 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:41.790343 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:41.790344 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:41.790669 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:41.790972 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:41.791203 (+ 231us) server_negotiation.cc:299] Negotiation successful
0806 11:36:41.791413 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260806 11:36:41.796173 27978 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:36:41.794847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54192 (local address 127.21.14.193:44191)
0806 11:36:41.795011 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:41.795014 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:41.795091 (+ 77us) server_negotiation.cc:407] Connection header received
0806 11:36:41.795320 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:41.795323 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:41.795368 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:41.795473 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:41.796032 (+ 559us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54192: BlockingRecv error: recv got EOF from 127.0.0.1:54192 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260806 11:36:41.796211 21563 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/krb5kdc/krb5cc' of type FILE
Aug 06 11:36:41 dist-test-slave-7c35 krb5kdc[27442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016201, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:36:41.797472 21563 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 11:36:41 dist-test-slave-7c35 krb5kdc[27442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016201, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:36:41.798754 21563 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 11:36:41 dist-test-slave-7c35 krb5kdc[27442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016201, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.14.193@KRBTEST.COM
I20260806 11:36:41.809959 27978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:41.800041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54206 (local address 127.21.14.193:44191)
0806 11:36:41.800311 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:41.800318 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:41.800342 (+ 24us) server_negotiation.cc:407] Connection header received
0806 11:36:41.800422 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:41.800425 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:41.800470 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:41.800535 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:41.801114 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:41.801763 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:41.802692 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:41.803080 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:41.803178 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:41.805949 (+ 2771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:41.805952 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:41.805954 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:41.805988 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:41.807836 (+ 1848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:41.808340 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:41.808344 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:41.808345 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:41.808435 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:41.808853 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:41.808858 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:41.808859 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:41.809128 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:41.809268 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:41.809565 (+ 297us) server_negotiation.cc:299] Negotiation successful
0806 11:36:41.809741 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260806 11:36:41.813087 27978 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:36:41.811963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54218 (local address 127.21.14.193:44191)
0806 11:36:41.812143 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:41.812146 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:41.812196 (+ 50us) server_negotiation.cc:407] Connection header received
0806 11:36:41.812319 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:41.812321 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:41.812360 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:41.812455 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:41.812949 (+ 494us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54218: BlockingRecv error: recv got EOF from 127.0.0.1:54218 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260806 11:36:41.813098 21563 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786016169045816-21563-0/krb5kdc/krb5cc' of type FILE
Aug 06 11:36:41 dist-test-slave-7c35 krb5kdc[27442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016201, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:36:41.814265 21563 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 11:36:41 dist-test-slave-7c35 krb5kdc[27442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016201, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:36:41.815272 21563 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 11:36:41 dist-test-slave-7c35 krb5kdc[27442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016201, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.14.193@KRBTEST.COM
I20260806 11:36:41.824821 27978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:41.816271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54228 (local address 127.21.14.193:44191)
0806 11:36:41.816423 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:41.816428 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:41.816548 (+ 120us) server_negotiation.cc:407] Connection header received
0806 11:36:41.816696 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:41.816701 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:41.816749 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:41.816840 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:41.817431 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:41.817946 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:41.818809 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:41.819102 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:41.819217 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:41.821614 (+ 2397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:41.821617 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:41.821619 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:41.821646 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:41.823072 (+ 1426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:41.823522 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:41.823526 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:41.823528 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:41.823623 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:41.823913 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:41.823917 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:41.823919 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:41.824114 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:41.824284 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:41.824515 (+ 231us) server_negotiation.cc:299] Negotiation successful
0806 11:36:41.824642 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 11:36:41.826112 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 27554
I20260806 11:36:41.834502 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 27694
I20260806 11:36:41.842065 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 27833
I20260806 11:36:41.851015 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 27455
2026-08-06T11:36:41Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2390 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 11:36:41 dist-test-slave-7c35 krb5kdc[27991](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 11:36:41 dist-test-slave-7c35 krb5kdc[27991](info): set up 2 sockets
Aug 06 11:36:41 dist-test-slave-7c35 krb5kdc[27991](info): commencing operation
krb5kdc: starting...
W20260806 11:36:42.942653 21563 mini_kdc.cc:121] Time spent starting KDC: real 1.067s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Aug 06 11:36:43 dist-test-slave-7c35 krb5kdc[27991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016203, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T11:36:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T11:36:43Z Disabled control of system clock
WARNING: no policy specified for oryx/127.21.14.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.14.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.14.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/krb5kdc/oryx_127.21.14.254.keytab.
Entry for principal oryx/127.21.14.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/krb5kdc/oryx_127.21.14.254.keytab.
I20260806 11:36:43.088972 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:46157
--webserver_interface=127.21.14.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/krb5kdc/oryx_127.21.14.254.keytab
--principal=oryx/127.21.14.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:43771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.14.254:46157
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:36:43.212144 28004 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:36:43.212549 28004 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:36:43.212651 28004 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:36:43.216903 28004 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 11:36:43.216979 28004 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:36:43.217000 28004 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:36:43.217017 28004 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 11:36:43.217033 28004 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 11:36:43.222702 28004 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:43771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.14.254:46157
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/krb5kdc/oryx_127.21.14.254.keytab
--principal=oryx/127.21.14.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:46157
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.14.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28004
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:36:43.223930 28004 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:36:43.225011 28004 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:36:43.234555 28010 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:43.234700 28004 server_base.cc:1034] running on GCE node
W20260806 11:36:43.234761 28013 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:43.234555 28011 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:43.235702 28004 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:36:43.237057 28004 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:36:43.238308 28004 hybrid_clock.cc:648] HybridClock initialized: now 1786016203238273 us; error 49 us; skew 500 ppm
Aug 06 11:36:43 dist-test-slave-7c35 krb5kdc[27991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016203, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.14.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:36:43.241802 28004 init.cc:377] Logged in from keytab as oryx/127.21.14.254@KRBTEST.COM (short username oryx)
I20260806 11:36:43.243424 28004 webserver.cc:460] Webserver started at http://127.21.14.254:32771/ using document root <none> and password file <none>
I20260806 11:36:43.244215 28004 fs_manager.cc:359] Metadata directory not provided
I20260806 11:36:43.244338 28004 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:36:43.244593 28004 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:36:43.247160 28004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/master-0/data/instance:
uuid: "f90d94258dce47129eb3ffe48021a93f"
format_stamp: "Formatted at 2026-08-06 11:36:43 on dist-test-slave-7c35"
server_key: "287511072eca31aa32bbb379ad20efa0"
server_key_iv: "0479278780dd46e915ca7414190527fd"
server_key_version: "encryptionkey@0"
I20260806 11:36:43.247851 28004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/master-0/wal/instance:
uuid: "f90d94258dce47129eb3ffe48021a93f"
format_stamp: "Formatted at 2026-08-06 11:36:43 on dist-test-slave-7c35"
server_key: "287511072eca31aa32bbb379ad20efa0"
server_key_iv: "0479278780dd46e915ca7414190527fd"
server_key_version: "encryptionkey@0"
I20260806 11:36:43.252411 28004 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 11:36:43.255577 28020 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:43.257081 28004 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 11:36:43.257263 28004 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/master-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/master-0/wal
uuid: "f90d94258dce47129eb3ffe48021a93f"
format_stamp: "Formatted at 2026-08-06 11:36:43 on dist-test-slave-7c35"
server_key: "287511072eca31aa32bbb379ad20efa0"
server_key_iv: "0479278780dd46e915ca7414190527fd"
server_key_version: "encryptionkey@0"
I20260806 11:36:43.257421 28004 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:36:43.270159 28004 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:36:43.273721 28004 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:36:43.273962 28004 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:36:43.282215 28004 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.254:46157
I20260806 11:36:43.282565 28074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.254:46157 every 8 connection(s)
I20260806 11:36:43.283464 28004 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
I20260806 11:36:43.286747 28075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:43.287909 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 28004
I20260806 11:36:43.288062 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/master-0/wal/instance
I20260806 11:36:43.288424 21563 external_mini_cluster.cc:1428] Setting key 025f3b2d04e01b8018919953870ac58a
I20260806 11:36:43.294282 28075 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f90d94258dce47129eb3ffe48021a93f: Bootstrap starting.
I20260806 11:36:43.297561 28075 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f90d94258dce47129eb3ffe48021a93f: Neither blocks nor log segments found. Creating new log.
Aug 06 11:36:43 dist-test-slave-7c35 krb5kdc[27991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016203, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM
I20260806 11:36:43.298519 28075 log.cc:824] T 00000000000000000000000000000000 P f90d94258dce47129eb3ffe48021a93f: Log is configured to *not* fsync() on all Append() calls
I20260806 11:36:43.300757 28075 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f90d94258dce47129eb3ffe48021a93f: No bootstrap required, opened a new log
I20260806 11:36:43.304234 28078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:43.289987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40518 (local address 127.21.14.254:46157)
0806 11:36:43.290519 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:43.290537 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:43.290580 (+ 43us) server_negotiation.cc:407] Connection header received
0806 11:36:43.291334 (+ 754us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:43.291357 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:43.291742 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:43.292052 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:43.293034 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:43.294072 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:43.295274 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:43.295807 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:43.295941 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:43.299075 (+ 3134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:43.299081 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:43.299098 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:43.299131 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:43.301048 (+ 1917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:43.301561 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:43.301567 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:43.301573 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:43.301700 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:43.302023 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:43.302033 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:43.302034 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:43.302490 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:43.302719 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:43.303042 (+ 323us) server_negotiation.cc:299] Negotiation successful
0806 11:36:43.303391 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":166,"threads_started":1}
I20260806 11:36:43.304839 28075 raft_consensus.cc:348] T 00000000000000000000000000000000 P f90d94258dce47129eb3ffe48021a93f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f90d94258dce47129eb3ffe48021a93f" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 46157 } }
I20260806 11:36:43.305130 28075 raft_consensus.cc:374] T 00000000000000000000000000000000 P f90d94258dce47129eb3ffe48021a93f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:43.305197 28075 raft_consensus.cc:729] T 00000000000000000000000000000000 P f90d94258dce47129eb3ffe48021a93f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f90d94258dce47129eb3ffe48021a93f, State: Initialized, Role: FOLLOWER
I20260806 11:36:43.305724 28075 consensus_queue.cc:260] T 00000000000000000000000000000000 P f90d94258dce47129eb3ffe48021a93f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f90d94258dce47129eb3ffe48021a93f" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 46157 } }
I20260806 11:36:43.305910 28075 raft_consensus.cc:388] T 00000000000000000000000000000000 P f90d94258dce47129eb3ffe48021a93f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 11:36:43.305992 28075 raft_consensus.cc:482] T 00000000000000000000000000000000 P f90d94258dce47129eb3ffe48021a93f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 11:36:43.306123 28075 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f90d94258dce47129eb3ffe48021a93f [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:36:43.307345 28075 raft_consensus.cc:504] T 00000000000000000000000000000000 P f90d94258dce47129eb3ffe48021a93f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f90d94258dce47129eb3ffe48021a93f" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 46157 } }
I20260806 11:36:43.307731 28075 leader_election.cc:302] T 00000000000000000000000000000000 P f90d94258dce47129eb3ffe48021a93f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f90d94258dce47129eb3ffe48021a93f; no voters:
I20260806 11:36:43.308108 28075 leader_election.cc:288] T 00000000000000000000000000000000 P f90d94258dce47129eb3ffe48021a93f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 11:36:43.308233 28080 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f90d94258dce47129eb3ffe48021a93f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:36:43.308588 28080 raft_consensus.cc:686] T 00000000000000000000000000000000 P f90d94258dce47129eb3ffe48021a93f [term 1 LEADER]: Becoming Leader. State: Replica: f90d94258dce47129eb3ffe48021a93f, State: Running, Role: LEADER
I20260806 11:36:43.308959 28080 consensus_queue.cc:237] T 00000000000000000000000000000000 P f90d94258dce47129eb3ffe48021a93f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f90d94258dce47129eb3ffe48021a93f" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 46157 } }
I20260806 11:36:43.309826 28075 sys_catalog.cc:564] T 00000000000000000000000000000000 P f90d94258dce47129eb3ffe48021a93f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 11:36:43.311040 28081 sys_catalog.cc:455] T 00000000000000000000000000000000 P f90d94258dce47129eb3ffe48021a93f [sys.catalog]: SysCatalogTable state changed. Reason: New leader f90d94258dce47129eb3ffe48021a93f. Latest consensus state: current_term: 1 leader_uuid: "f90d94258dce47129eb3ffe48021a93f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f90d94258dce47129eb3ffe48021a93f" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 46157 } } }
I20260806 11:36:43.311192 28081 sys_catalog.cc:458] T 00000000000000000000000000000000 P f90d94258dce47129eb3ffe48021a93f [sys.catalog]: This master's current role is: LEADER
I20260806 11:36:43.311640 28089 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 11:36:43.311643 28082 sys_catalog.cc:455] T 00000000000000000000000000000000 P f90d94258dce47129eb3ffe48021a93f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f90d94258dce47129eb3ffe48021a93f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f90d94258dce47129eb3ffe48021a93f" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 46157 } } }
I20260806 11:36:43.311898 28082 sys_catalog.cc:458] T 00000000000000000000000000000000 P f90d94258dce47129eb3ffe48021a93f [sys.catalog]: This master's current role is: LEADER
I20260806 11:36:43.315184 28089 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 11:36:43.322517 28089 catalog_manager.cc:1347] Generated new cluster ID: f64fef229c0a4d3588ddb9a279aad6dd
I20260806 11:36:43.322624 28089 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 11:36:43.331951 28089 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 11:36:43.333191 28089 catalog_manager.cc:1504] Loading token signing keys...
I20260806 11:36:43.343787 28089 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f90d94258dce47129eb3ffe48021a93f: Generated new TSK 0
I20260806 11:36:43.344640 28089 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.21.14.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.14.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.14.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/krb5kdc/oryx_127.21.14.193.keytab.
Entry for principal oryx/127.21.14.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/krb5kdc/oryx_127.21.14.193.keytab.
I20260806 11:36:43.386695 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.193:0
--local_ip_for_outbound_sockets=127.21.14.193
--webserver_interface=127.21.14.193
--webserver_port=0
--tserver_master_addrs=127.21.14.254:46157
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/krb5kdc/oryx_127.21.14.193.keytab
--principal=oryx/127.21.14.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:43771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:36:43.512012 28101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:36:43.512324 28101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:36:43.512398 28101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:36:43.517036 28101 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:36:43.517131 28101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:36:43.517197 28101 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.193
I20260806 11:36:43.523123 28101 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:43771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/krb5kdc/oryx_127.21.14.193.keytab
--principal=oryx/127.21.14.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.14.193
--webserver_port=0
--tserver_master_addrs=127.21.14.254:46157
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28101
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.193
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:36:43.524472 28101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:36:43.525759 28101 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:36:43.540781 28106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:43.541007 28101 server_base.cc:1034] running on GCE node
W20260806 11:36:43.541014 28107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:43.540854 28110 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:43.541714 28101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:36:43.542553 28101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:36:43.543773 28101 hybrid_clock.cc:648] HybridClock initialized: now 1786016203543751 us; error 44 us; skew 500 ppm
Aug 06 11:36:43 dist-test-slave-7c35 krb5kdc[27991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016203, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.14.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:36:43.547240 28101 init.cc:377] Logged in from keytab as oryx/127.21.14.193@KRBTEST.COM (short username oryx)
I20260806 11:36:43.548338 28101 webserver.cc:460] Webserver started at http://127.21.14.193:42231/ using document root <none> and password file <none>
I20260806 11:36:43.549007 28101 fs_manager.cc:359] Metadata directory not provided
I20260806 11:36:43.549079 28101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:36:43.549324 28101 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:36:43.551873 28101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-0/data/instance:
uuid: "cb38f9cc24a2413289605d92553b0b60"
format_stamp: "Formatted at 2026-08-06 11:36:43 on dist-test-slave-7c35"
server_key: "507165f35b358b0309cc46aab5f69e0f"
server_key_iv: "22d64c7195f132b1747e50eacbecd45d"
server_key_version: "encryptionkey@0"
I20260806 11:36:43.552847 28101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-0/wal/instance:
uuid: "cb38f9cc24a2413289605d92553b0b60"
format_stamp: "Formatted at 2026-08-06 11:36:43 on dist-test-slave-7c35"
server_key: "507165f35b358b0309cc46aab5f69e0f"
server_key_iv: "22d64c7195f132b1747e50eacbecd45d"
server_key_version: "encryptionkey@0"
I20260806 11:36:43.557667 28101 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 11:36:43.561743 28118 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:43.564241 28101 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260806 11:36:43.564435 28101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-0/wal
uuid: "cb38f9cc24a2413289605d92553b0b60"
format_stamp: "Formatted at 2026-08-06 11:36:43 on dist-test-slave-7c35"
server_key: "507165f35b358b0309cc46aab5f69e0f"
server_key_iv: "22d64c7195f132b1747e50eacbecd45d"
server_key_version: "encryptionkey@0"
I20260806 11:36:43.564602 28101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:36:43.582801 28101 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:36:43.586926 28101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:36:43.587193 28101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:36:43.587954 28101 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:36:43.589003 28101 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:36:43.589063 28101 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:43.589110 28101 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:36:43.589126 28101 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:43.600481 28101 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.193:38635
I20260806 11:36:43.600684 28238 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.193:38635 every 8 connection(s)
I20260806 11:36:43.601621 28101 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
I20260806 11:36:43.606509 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 28101
I20260806 11:36:43.606670 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-0/wal/instance
I20260806 11:36:43.607074 21563 external_mini_cluster.cc:1428] Setting key 7a5b4fd9711fa12923e66c809fdcb425
Aug 06 11:36:43 dist-test-slave-7c35 krb5kdc[27991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016203, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.14.193@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM
WARNING: no policy specified for oryx/127.21.14.194@KRBTEST.COM; defaulting to no policy
I20260806 11:36:43.621619 28078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:43.603943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:39463 (local address 127.21.14.254:46157)
0806 11:36:43.604151 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:43.604155 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:43.605147 (+ 992us) server_negotiation.cc:407] Connection header received
0806 11:36:43.606974 (+ 1827us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:43.606978 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:43.607045 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:43.607157 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:43.608592 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:43.609375 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:43.611071 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:43.611416 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:43.611516 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:43.615481 (+ 3965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:43.615487 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:43.615490 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:43.615546 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:43.617905 (+ 2359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:43.619007 (+ 1102us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:43.619011 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:43.619013 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:43.619155 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:43.619761 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:43.619765 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:43.619767 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:43.620057 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:43.620172 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:43.621226 (+ 1054us) server_negotiation.cc:299] Negotiation successful
0806 11:36:43.621416 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260806 11:36:43.622951 28241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:43.604376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:46157 (local address 127.21.14.193:39463)
0806 11:36:43.604949 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:43.605006 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:43.606643 (+ 1637us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:43.607392 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:43.607403 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:43.607855 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:43.608412 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:43.608428 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:43.609593 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:43.609596 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:43.610857 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:43.610902 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:43.611634 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:43.611641 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:43.611926 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:43.612797 (+ 871us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:43.612819 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:43.615107 (+ 2288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:43.618156 (+ 3049us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:43.618167 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:43.618193 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:43.618811 (+ 618us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:43.619298 (+ 487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:43.619302 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:43.619304 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:43.619582 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:43.620280 (+ 698us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:43.620289 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:43.620735 (+ 446us) client_negotiation.cc:769] Sending connection context
0806 11:36:43.620993 (+ 258us) client_negotiation.cc:240] Negotiation successful
0806 11:36:43.621524 (+ 531us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":140,"threads_started":1}
I20260806 11:36:43.625036 28239 heartbeater.cc:347] Connected to a master server at 127.21.14.254:46157
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.14.194@KRBTEST.COM" created.
I20260806 11:36:43.625362 28239 heartbeater.cc:464] Registering TS with master...
I20260806 11:36:43.626065 28239 heartbeater.cc:511] Master 127.21.14.254:46157 requested a full tablet report, sending...
I20260806 11:36:43.628153 28038 ts_manager.cc:194] Registered new tserver with Master: cb38f9cc24a2413289605d92553b0b60 (127.21.14.193:38635)
I20260806 11:36:43.629511 28038 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.14.193@KRBTEST.COM'} at 127.21.14.193:39463
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.14.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/krb5kdc/oryx_127.21.14.194.keytab.
Entry for principal oryx/127.21.14.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/krb5kdc/oryx_127.21.14.194.keytab.
I20260806 11:36:43.647275 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.194:0
--local_ip_for_outbound_sockets=127.21.14.194
--webserver_interface=127.21.14.194
--webserver_port=0
--tserver_master_addrs=127.21.14.254:46157
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/krb5kdc/oryx_127.21.14.194.keytab
--principal=oryx/127.21.14.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:43771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:36:43.773145 28244 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:36:43.773489 28244 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:36:43.773545 28244 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:36:43.778332 28244 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:36:43.778426 28244 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:36:43.778491 28244 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.194
I20260806 11:36:43.784564 28244 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:43771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/krb5kdc/oryx_127.21.14.194.keytab
--principal=oryx/127.21.14.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.14.194
--webserver_port=0
--tserver_master_addrs=127.21.14.254:46157
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28244
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.194
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:36:43.785841 28244 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:36:43.787062 28244 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:36:43.798516 28250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:43.798530 28249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:43.798429 28252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:43.798754 28244 server_base.cc:1034] running on GCE node
I20260806 11:36:43.799353 28244 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:36:43.800158 28244 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:36:43.801388 28244 hybrid_clock.cc:648] HybridClock initialized: now 1786016203801365 us; error 38 us; skew 500 ppm
Aug 06 11:36:43 dist-test-slave-7c35 krb5kdc[27991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016203, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.14.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:36:43.804695 28244 init.cc:377] Logged in from keytab as oryx/127.21.14.194@KRBTEST.COM (short username oryx)
I20260806 11:36:43.805792 28244 webserver.cc:460] Webserver started at http://127.21.14.194:41511/ using document root <none> and password file <none>
I20260806 11:36:43.806479 28244 fs_manager.cc:359] Metadata directory not provided
I20260806 11:36:43.806607 28244 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:36:43.806951 28244 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:36:43.809372 28244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-1/data/instance:
uuid: "0e4d7e5ba5744b47a854f00460dac7aa"
format_stamp: "Formatted at 2026-08-06 11:36:43 on dist-test-slave-7c35"
server_key: "a26c1d5d96ca940da624a6f5dee83708"
server_key_iv: "c03bd8cd911a068dff7e9bc220adef4e"
server_key_version: "encryptionkey@0"
I20260806 11:36:43.810140 28244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-1/wal/instance:
uuid: "0e4d7e5ba5744b47a854f00460dac7aa"
format_stamp: "Formatted at 2026-08-06 11:36:43 on dist-test-slave-7c35"
server_key: "a26c1d5d96ca940da624a6f5dee83708"
server_key_iv: "c03bd8cd911a068dff7e9bc220adef4e"
server_key_version: "encryptionkey@0"
I20260806 11:36:43.815181 28244 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260806 11:36:43.818354 28259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:43.820048 28244 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 11:36:43.820255 28244 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-1/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-1/wal
uuid: "0e4d7e5ba5744b47a854f00460dac7aa"
format_stamp: "Formatted at 2026-08-06 11:36:43 on dist-test-slave-7c35"
server_key: "a26c1d5d96ca940da624a6f5dee83708"
server_key_iv: "c03bd8cd911a068dff7e9bc220adef4e"
server_key_version: "encryptionkey@0"
I20260806 11:36:43.820415 28244 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:36:43.843014 28244 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:36:43.847672 28244 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:36:43.847955 28244 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:36:43.848690 28244 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:36:43.849696 28244 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:36:43.849747 28244 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:43.849819 28244 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:36:43.849853 28244 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:43.859987 28244 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.194:36707
I20260806 11:36:43.860029 28382 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.194:36707 every 8 connection(s)
I20260806 11:36:43.861101 28244 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
I20260806 11:36:43.866119 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 28244
I20260806 11:36:43.866268 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-1/wal/instance
I20260806 11:36:43.866671 21563 external_mini_cluster.cc:1428] Setting key 88463777bce0be278c0e8cdff4c21d22
Aug 06 11:36:43 dist-test-slave-7c35 krb5kdc[27991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016203, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.14.194@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM
WARNING: no policy specified for oryx/127.21.14.195@KRBTEST.COM; defaulting to no policy
I20260806 11:36:43.878227 28078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:43.863595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:59375 (local address 127.21.14.254:46157)
0806 11:36:43.863785 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:43.863791 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:43.864882 (+ 1091us) server_negotiation.cc:407] Connection header received
0806 11:36:43.866455 (+ 1573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:43.866461 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:43.866535 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:43.866647 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:43.868313 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:43.868948 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:43.870345 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:43.870589 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:43.870710 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:43.873986 (+ 3276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:43.873996 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:43.873998 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:43.874031 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:43.875696 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:43.876349 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:43.876355 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:43.876362 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:43.876450 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:43.876848 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:43.876852 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:43.876854 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:43.877086 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:43.877205 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:43.877884 (+ 679us) server_negotiation.cc:299] Negotiation successful
0806 11:36:43.878049 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 11:36:43.879396 28385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:43.864095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:46157 (local address 127.21.14.194:59375)
0806 11:36:43.864684 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:43.864738 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:43.866096 (+ 1358us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:43.866931 (+ 835us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:43.866940 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:43.867455 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:43.868139 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:43.868163 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:43.869111 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:43.869114 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:43.870201 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:43.870214 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:43.870770 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:43.870774 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:43.871066 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:43.871791 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:43.871811 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:43.873711 (+ 1900us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:43.875868 (+ 2157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:43.875876 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:43.875907 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:43.876225 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:43.876558 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:43.876562 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:43.876564 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:43.876739 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:43.877206 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:43.877215 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:43.877542 (+ 327us) client_negotiation.cc:769] Sending connection context
0806 11:36:43.877833 (+ 291us) client_negotiation.cc:240] Negotiation successful
0806 11:36:43.878234 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":177,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.14.195@KRBTEST.COM" created.
I20260806 11:36:43.880977 28383 heartbeater.cc:347] Connected to a master server at 127.21.14.254:46157
I20260806 11:36:43.881315 28383 heartbeater.cc:464] Registering TS with master...
I20260806 11:36:43.881934 28383 heartbeater.cc:511] Master 127.21.14.254:46157 requested a full tablet report, sending...
I20260806 11:36:43.883574 28038 ts_manager.cc:194] Registered new tserver with Master: 0e4d7e5ba5744b47a854f00460dac7aa (127.21.14.194:36707)
I20260806 11:36:43.884559 28038 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.14.194@KRBTEST.COM'} at 127.21.14.194:59375
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.14.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/krb5kdc/oryx_127.21.14.195.keytab.
Entry for principal oryx/127.21.14.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/krb5kdc/oryx_127.21.14.195.keytab.
I20260806 11:36:43.903947 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.195:0
--local_ip_for_outbound_sockets=127.21.14.195
--webserver_interface=127.21.14.195
--webserver_port=0
--tserver_master_addrs=127.21.14.254:46157
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/krb5kdc/oryx_127.21.14.195.keytab
--principal=oryx/127.21.14.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:43771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:36:44.030730 28388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:36:44.031123 28388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:36:44.031172 28388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:36:44.035604 28388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:36:44.035698 28388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:36:44.035764 28388 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.195
I20260806 11:36:44.041682 28388 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:43771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/krb5kdc/oryx_127.21.14.195.keytab
--principal=oryx/127.21.14.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.14.195
--webserver_port=0
--tserver_master_addrs=127.21.14.254:46157
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28388
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.195
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:36:44.043025 28388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:36:44.044193 28388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:36:44.055217 28394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:44.055207 28393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:44.055207 28396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:44.055729 28388 server_base.cc:1034] running on GCE node
I20260806 11:36:44.056291 28388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:36:44.057051 28388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:36:44.058300 28388 hybrid_clock.cc:648] HybridClock initialized: now 1786016204058279 us; error 32 us; skew 500 ppm
Aug 06 11:36:44 dist-test-slave-7c35 krb5kdc[27991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016204, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.14.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:36:44.062572 28388 init.cc:377] Logged in from keytab as oryx/127.21.14.195@KRBTEST.COM (short username oryx)
I20260806 11:36:44.063850 28388 webserver.cc:460] Webserver started at http://127.21.14.195:42525/ using document root <none> and password file <none>
I20260806 11:36:44.064564 28388 fs_manager.cc:359] Metadata directory not provided
I20260806 11:36:44.064677 28388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:36:44.064941 28388 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:36:44.067494 28388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-2/data/instance:
uuid: "98033857a04c4ac794f0feeead2ddc3d"
format_stamp: "Formatted at 2026-08-06 11:36:44 on dist-test-slave-7c35"
server_key: "fbe86ef7178c7583cb7bf7e3a5eed884"
server_key_iv: "f317588ef27887ad0b713f9a7fe4bc71"
server_key_version: "encryptionkey@0"
I20260806 11:36:44.068369 28388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-2/wal/instance:
uuid: "98033857a04c4ac794f0feeead2ddc3d"
format_stamp: "Formatted at 2026-08-06 11:36:44 on dist-test-slave-7c35"
server_key: "fbe86ef7178c7583cb7bf7e3a5eed884"
server_key_iv: "f317588ef27887ad0b713f9a7fe4bc71"
server_key_version: "encryptionkey@0"
I20260806 11:36:44.073299 28388 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 11:36:44.076817 28403 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:44.078476 28388 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 11:36:44.078672 28388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-2/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-2/wal
uuid: "98033857a04c4ac794f0feeead2ddc3d"
format_stamp: "Formatted at 2026-08-06 11:36:44 on dist-test-slave-7c35"
server_key: "fbe86ef7178c7583cb7bf7e3a5eed884"
server_key_iv: "f317588ef27887ad0b713f9a7fe4bc71"
server_key_version: "encryptionkey@0"
I20260806 11:36:44.078822 28388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:36:44.099531 28388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:36:44.104461 28388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:36:44.104732 28388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:36:44.105683 28388 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:36:44.106779 28388 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:36:44.106904 28388 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260806 11:36:44.107023 28388 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:36:44.107067 28388 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:44.121061 28388 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.195:41935
I20260806 11:36:44.121088 28531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.195:41935 every 8 connection(s)
I20260806 11:36:44.122394 28388 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
I20260806 11:36:44.123771 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 28388
I20260806 11:36:44.123914 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-2/wal/instance
I20260806 11:36:44.124300 21563 external_mini_cluster.cc:1428] Setting key d1c244dd3da65fa9e151ddc98fc4f2ae
Aug 06 11:36:44 dist-test-slave-7c35 krb5kdc[27991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016204, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.14.195@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM
I20260806 11:36:44.143628 28078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:44.124985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:36885 (local address 127.21.14.254:46157)
0806 11:36:44.125162 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:44.125167 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:44.126537 (+ 1370us) server_negotiation.cc:407] Connection header received
0806 11:36:44.128662 (+ 2125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:44.128666 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:44.128736 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:44.129184 (+ 448us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:44.131013 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:44.131561 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:44.133003 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:44.133255 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:44.133392 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:44.137936 (+ 4544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:44.137939 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:44.137942 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:44.137975 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:44.140218 (+ 2243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:44.141293 (+ 1075us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:44.141297 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:44.141298 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:44.141414 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:44.141963 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:44.141966 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:44.141968 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:44.142228 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:44.142319 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:44.143325 (+ 1006us) server_negotiation.cc:299] Negotiation successful
0806 11:36:44.143478 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 11:36:44.144600 28534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:44.125653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:46157 (local address 127.21.14.195:36885)
0806 11:36:44.126367 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:44.126428 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:44.128322 (+ 1894us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:44.129493 (+ 1171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:44.129503 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:44.130137 (+ 634us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:44.130742 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:44.130760 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:44.131727 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:44.131736 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:44.132764 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:44.132778 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:44.133464 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:44.133469 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:44.133694 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:44.134461 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:44.134481 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:44.137540 (+ 3059us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:44.140436 (+ 2896us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:44.140444 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:44.140465 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:44.140855 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:44.141565 (+ 710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:44.141568 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:44.141570 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:44.141787 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:44.142492 (+ 705us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:44.142501 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:44.142922 (+ 421us) client_negotiation.cc:769] Sending connection context
0806 11:36:44.143157 (+ 235us) client_negotiation.cc:240] Negotiation successful
0806 11:36:44.143507 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":206,"threads_started":1}
I20260806 11:36:44.146461 28532 heartbeater.cc:347] Connected to a master server at 127.21.14.254:46157
I20260806 11:36:44.146806 28532 heartbeater.cc:464] Registering TS with master...
I20260806 11:36:44.147516 28532 heartbeater.cc:511] Master 127.21.14.254:46157 requested a full tablet report, sending...
I20260806 11:36:44.149015 28038 ts_manager.cc:194] Registered new tserver with Master: 98033857a04c4ac794f0feeead2ddc3d (127.21.14.195:41935)
I20260806 11:36:44.149694 28038 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.14.195@KRBTEST.COM'} at 127.21.14.195:36885
I20260806 11:36:44.152362 21563 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.21.14.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.14.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.14.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/krb5kdc/oryx_127.21.14.253.keytab.
Entry for principal oryx/127.21.14.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/krb5kdc/oryx_127.21.14.253.keytab.
I20260806 11:36:44.198243 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.253:43931
--webserver_interface=127.21.14.253
--webserver_port=0
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/krb5kdc/oryx_127.21.14.253.keytab
--principal=oryx/127.21.14.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:43771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.14.254:46157,127.21.14.253:43931
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:36:44.338363 28537 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:36:44.338737 28537 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:36:44.338830 28537 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:36:44.343968 28537 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 11:36:44.344074 28537 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:36:44.344096 28537 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:36:44.344115 28537 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 11:36:44.344130 28537 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 11:36:44.350431 28537 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:43771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.21.14.254:46157,127.21.14.253:43931
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/krb5kdc/oryx_127.21.14.253.keytab
--principal=oryx/127.21.14.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.253:43931
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.21.14.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28537
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
W20260806 11:36:44.350760 28537 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.21.14.254:46157,127.21.14.253:43931'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260806 11:36:44.351905 28537 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:36:44.353145 28537 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:36:44.364559 28542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:44.364570 28543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:44.365125 28537 server_base.cc:1034] running on GCE node
W20260806 11:36:44.365082 28545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:44.366436 28537 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:36:44.368000 28537 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:36:44.369601 28537 hybrid_clock.cc:648] HybridClock initialized: now 1786016204369554 us; error 80 us; skew 500 ppm
Aug 06 11:36:44 dist-test-slave-7c35 krb5kdc[27991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016204, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.14.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:36:44.373769 28537 init.cc:377] Logged in from keytab as oryx/127.21.14.253@KRBTEST.COM (short username oryx)
I20260806 11:36:44.375180 28537 webserver.cc:460] Webserver started at http://127.21.14.253:37915/ using document root <none> and password file <none>
I20260806 11:36:44.375882 28537 fs_manager.cc:359] Metadata directory not provided
I20260806 11:36:44.376010 28537 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:36:44.376303 28537 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:36:44.378754 28537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/master-1/data/instance:
uuid: "b160eb10619648eeacfcc3d72bae9fd4"
format_stamp: "Formatted at 2026-08-06 11:36:44 on dist-test-slave-7c35"
server_key: "737a6fe322a75a315d0be2e1f364131b"
server_key_iv: "7d7ddd70c4b564586de219491d24d532"
server_key_version: "encryptionkey@0"
I20260806 11:36:44.379624 28537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/master-1/wal/instance:
uuid: "b160eb10619648eeacfcc3d72bae9fd4"
format_stamp: "Formatted at 2026-08-06 11:36:44 on dist-test-slave-7c35"
server_key: "737a6fe322a75a315d0be2e1f364131b"
server_key_iv: "7d7ddd70c4b564586de219491d24d532"
server_key_version: "encryptionkey@0"
I20260806 11:36:44.384743 28537 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 11:36:44.388652 28552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:44.390733 28537 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 11:36:44.391019 28537 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/master-1/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/master-1/wal
uuid: "b160eb10619648eeacfcc3d72bae9fd4"
format_stamp: "Formatted at 2026-08-06 11:36:44 on dist-test-slave-7c35"
server_key: "737a6fe322a75a315d0be2e1f364131b"
server_key_iv: "7d7ddd70c4b564586de219491d24d532"
server_key_version: "encryptionkey@0"
I20260806 11:36:44.391191 28537 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:36:44.405568 28537 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:36:44.410311 28537 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:36:44.410575 28537 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:36:44.420601 28537 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.253:43931
I20260806 11:36:44.420617 28606 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.253:43931 every 8 connection(s)
I20260806 11:36:44.421795 28537 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/master-1/data/info.pb
I20260806 11:36:44.425412 28607 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:44.429059 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 28537
I20260806 11:36:44.431416 28607 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.14.254" port: 46157 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 11:36:44 dist-test-slave-7c35 krb5kdc[27991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016203, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.14.253@KRBTEST.COM
Aug 06 11:36:44 dist-test-slave-7c35 krb5kdc[27991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016204, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.14.253@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM
I20260806 11:36:44.446581 28078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:44.433579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40530 (local address 127.21.14.254:46157)
0806 11:36:44.433741 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:44.433747 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:44.434171 (+ 424us) server_negotiation.cc:407] Connection header received
0806 11:36:44.434565 (+ 394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:44.434571 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:44.434648 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:44.434756 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:44.435688 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:44.436354 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:44.438102 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:44.438450 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:44.438575 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:44.442503 (+ 3928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:44.442507 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:44.442509 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:44.442546 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:44.444238 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:44.444844 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:44.444849 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:44.444851 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:44.444946 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:44.445275 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:44.445278 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:44.445280 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:44.445528 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:44.445645 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:44.446074 (+ 429us) server_negotiation.cc:299] Negotiation successful
0806 11:36:44.446406 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 11:36:44.447392 28611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:44.433489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:46157 (local address 127.0.0.1:40530)
0806 11:36:44.433980 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:44.434071 (+ 91us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:44.434284 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:44.434827 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:44.434833 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:44.435320 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:44.435505 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:44.435517 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:44.436532 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:44.436535 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:44.437911 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:44.437929 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:44.438692 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:44.438697 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:44.438787 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:44.439445 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:44.439472 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:44.442258 (+ 2786us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:44.444392 (+ 2134us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:44.444404 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:44.444412 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:44.444721 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:44.445063 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:44.445067 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:44.445069 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:44.445171 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:44.445648 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:44.445658 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:44.445820 (+ 162us) client_negotiation.cc:769] Sending connection context
0806 11:36:44.445998 (+ 178us) client_negotiation.cc:240] Negotiation successful
0806 11:36:44.446413 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"mutex_wait_us":101,"spinlock_wait_cycles":4864,"thread_start_us":143,"threads_started":1}
I20260806 11:36:44.447392 28610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:44.431070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34178 (local address 127.21.14.253:43931)
0806 11:36:44.431633 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:44.431654 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:44.431700 (+ 46us) server_negotiation.cc:407] Connection header received
0806 11:36:44.432631 (+ 931us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:44.432654 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:44.433120 (+ 466us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:44.433545 (+ 425us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:44.434649 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:44.435948 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:44.437320 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:44.438096 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:44.438316 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:44.441750 (+ 3434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:44.441760 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:44.441782 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:44.441821 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:44.443782 (+ 1961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:44.444391 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:44.444398 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:44.444404 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:44.444567 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:44.444900 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:44.444904 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:44.444906 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:44.445445 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:44.445706 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:44.445967 (+ 261us) server_negotiation.cc:299] Negotiation successful
0806 11:36:44.446314 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"spinlock_wait_cycles":3456,"thread_start_us":186,"threads_started":1}
I20260806 11:36:44.449747 28607 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.14.253" port: 43931 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 11:36:44 dist-test-slave-7c35 krb5kdc[27991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016204, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.14.253@KRBTEST.COM for oryx/127.21.14.253@KRBTEST.COM
I20260806 11:36:44.460642 28611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:44.450381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.253:43931 (local address 127.0.0.1:34192)
0806 11:36:44.450526 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:44.450539 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:44.450629 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:44.450949 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:44.450952 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:44.451147 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:44.451249 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:44.451255 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:44.452117 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:44.452120 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:44.453475 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:44.453491 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:44.454059 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:44.454062 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:44.454154 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:44.454721 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:44.454734 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:44.456552 (+ 1818us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:44.458818 (+ 2266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:44.458822 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:44.458824 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:44.459138 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:44.459600 (+ 462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:44.459603 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:44.459605 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:44.459715 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:44.460162 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:44.460165 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:44.460259 (+ 94us) client_negotiation.cc:769] Sending connection context
0806 11:36:44.460349 (+ 90us) client_negotiation.cc:240] Negotiation successful
0806 11:36:44.460483 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260806 11:36:44.460881 28610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:44.450500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34192 (local address 127.21.14.253:43931)
0806 11:36:44.450638 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:44.450647 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:44.450663 (+ 16us) server_negotiation.cc:407] Connection header received
0806 11:36:44.450698 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:44.450700 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:44.450736 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:44.450810 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:44.451409 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:44.451978 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:44.453685 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:44.453915 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:44.454013 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:44.456743 (+ 2730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:44.456747 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:44.456749 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:44.456776 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:44.458640 (+ 1864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:44.459302 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:44.459305 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:44.459306 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:44.459401 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:44.459833 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:44.459836 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:44.459837 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:44.460060 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:44.460148 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:44.460463 (+ 315us) server_negotiation.cc:299] Negotiation successful
0806 11:36:44.460609 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 11:36:44.464843 28607 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4: Bootstrap starting.
I20260806 11:36:44.467996 28607 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:44.468924 28607 log.cc:824] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4: Log is configured to *not* fsync() on all Append() calls
I20260806 11:36:44.471532 28607 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4: No bootstrap required, opened a new log
I20260806 11:36:44.474946 28607 raft_consensus.cc:348] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f90d94258dce47129eb3ffe48021a93f" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 46157 } } peers { permanent_uuid: "b160eb10619648eeacfcc3d72bae9fd4" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 43931 } }
I20260806 11:36:44.475227 28607 raft_consensus.cc:374] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:44.475265 28607 raft_consensus.cc:729] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b160eb10619648eeacfcc3d72bae9fd4, State: Initialized, Role: FOLLOWER
I20260806 11:36:44.475709 28607 consensus_queue.cc:260] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f90d94258dce47129eb3ffe48021a93f" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 46157 } } peers { permanent_uuid: "b160eb10619648eeacfcc3d72bae9fd4" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 43931 } }
I20260806 11:36:44.476256 28613 sys_catalog.cc:455] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f90d94258dce47129eb3ffe48021a93f" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 46157 } } peers { permanent_uuid: "b160eb10619648eeacfcc3d72bae9fd4" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 43931 } } }
I20260806 11:36:44.476402 28613 sys_catalog.cc:458] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 11:36:44.477200 28607 sys_catalog.cc:564] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 11:36:44.482797 28626 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 11:36:44.482992 28626 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 11:36:44.486254 28537 master_runner.cc:428] Detected that this master b160eb10619648eeacfcc3d72bae9fd4 is joining an existing cluster
I20260806 11:36:44.486389 28537 master_runner.cc:432] Initiating AddMaster RPC to add 127.21.14.253:43931
I20260806 11:36:44.499408 28610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:44.489613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34194 (local address 127.21.14.253:43931)
0806 11:36:44.489717 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:44.489721 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:44.489785 (+ 64us) server_negotiation.cc:407] Connection header received
0806 11:36:44.489902 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:44.489904 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:44.489940 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:44.490016 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:44.490733 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:44.491724 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:44.493082 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:44.493455 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:44.493598 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:44.495373 (+ 1775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:44.495377 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:44.495379 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:44.495402 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:44.497328 (+ 1926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:44.497926 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:44.497931 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:44.497934 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:44.498030 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:44.498353 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:44.498357 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:44.498359 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:44.498606 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:44.498706 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:44.499070 (+ 364us) server_negotiation.cc:299] Negotiation successful
0806 11:36:44.499244 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":71,"server-negotiator.queue_time_us":50}
I20260806 11:36:44.499449 28078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:44.488994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40534 (local address 127.21.14.254:46157)
0806 11:36:44.489150 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:44.489155 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:44.489607 (+ 452us) server_negotiation.cc:407] Connection header received
0806 11:36:44.489871 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:44.489877 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:44.489941 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:44.490016 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:44.490722 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:44.491714 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:44.492882 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:44.493500 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:44.493611 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:44.495440 (+ 1829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:44.495443 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:44.495446 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:44.495479 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:44.497319 (+ 1840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:44.498059 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:44.498063 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:44.498066 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:44.498169 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:44.498465 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:44.498468 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:44.498470 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:44.498726 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:44.498834 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:44.499141 (+ 307us) server_negotiation.cc:299] Negotiation successful
0806 11:36:44.499283 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 11:36:44.499421 28632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:44.489001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.253:43931 (local address 127.0.0.1:34194)
0806 11:36:44.489678 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:44.489695 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:44.489814 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:44.490087 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:44.490092 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:44.490410 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:44.490591 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:44.490600 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:44.491868 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:44.491872 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:44.492842 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:44.492854 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:44.493642 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:44.493649 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:44.493770 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:44.494505 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:44.494522 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:44.495211 (+ 689us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:44.497465 (+ 2254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:44.497468 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:44.497471 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:44.497815 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:44.498143 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:44.498146 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:44.498148 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:44.498261 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:44.498716 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:44.498719 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:44.498826 (+ 107us) client_negotiation.cc:769] Sending connection context
0806 11:36:44.499009 (+ 183us) client_negotiation.cc:240] Negotiation successful
0806 11:36:44.499188 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":606,"mutex_wait_us":43,"spinlock_wait_cycles":9088,"thread_start_us":301,"threads_started":1}
I20260806 11:36:44.500264 28633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:44.488937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:46157 (local address 127.0.0.1:40534)
0806 11:36:44.489475 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:44.489488 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:44.489701 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:44.490166 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:44.490168 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:44.490411 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:44.490560 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:44.490566 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:44.491911 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:44.491913 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:44.492692 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:44.492706 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:44.493763 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:44.493766 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:44.493868 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:44.494588 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:44.494600 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:44.495266 (+ 666us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:44.497470 (+ 2204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:44.497476 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:44.497481 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:44.497842 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:44.498270 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:44.498273 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:44.498275 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:44.498359 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:44.498813 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:44.498816 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:44.498925 (+ 109us) client_negotiation.cc:769] Sending connection context
0806 11:36:44.499688 (+ 763us) client_negotiation.cc:240] Negotiation successful
0806 11:36:44.499843 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"mutex_wait_us":95,"spinlock_wait_cycles":896,"thread_start_us":142,"threads_started":1}
Aug 06 11:36:44 dist-test-slave-7c35 krb5kdc[27991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016203, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.14.254@KRBTEST.COM for oryx/127.21.14.253@KRBTEST.COM
WARNING: no policy specified for oryx/127.21.14.252@KRBTEST.COM; defaulting to no policy
I20260806 11:36:44.516554 28610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:44.505946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34204 (local address 127.21.14.253:43931)
0806 11:36:44.506096 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:44.506101 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:44.506437 (+ 336us) server_negotiation.cc:407] Connection header received
0806 11:36:44.506697 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:44.506701 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:44.506762 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:44.506855 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:44.507641 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:44.508169 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:44.509175 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:44.509405 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:44.509529 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:44.512436 (+ 2907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:44.512440 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:44.512442 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:44.512471 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:44.514063 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:44.514677 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:44.514680 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:44.514684 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:44.514773 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:44.515244 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:44.515248 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:44.515253 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:44.515482 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:44.515599 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:44.516088 (+ 489us) server_negotiation.cc:299] Negotiation successful
0806 11:36:44.516365 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 11:36:44.516597 28635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:44.505898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.253:43931 (local address 127.0.0.1:34204)
0806 11:36:44.506298 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:44.506344 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:44.506533 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:44.506929 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:44.506937 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:44.507370 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:44.507503 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:44.507511 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:44.508297 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:44.508302 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:44.509025 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:44.509036 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:44.509544 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:44.509548 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:44.509709 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:44.510455 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:44.510470 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:44.512213 (+ 1743us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:44.514226 (+ 2013us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:44.514242 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:44.514251 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:44.514558 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:44.514919 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:44.514922 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:44.514924 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:44.515023 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:44.515623 (+ 600us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:44.515633 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:44.515829 (+ 196us) client_negotiation.cc:769] Sending connection context
0806 11:36:44.516012 (+ 183us) client_negotiation.cc:240] Negotiation successful
0806 11:36:44.516388 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":105,"threads_started":1}
I20260806 11:36:44.519151 28038 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.21.14.253:43931: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "b160eb10619648eeacfcc3d72bae9fd4"
member_type: NON_VOTER
last_known_addr {
host: "127.21.14.253"
port: 43931
}
attrs {
promote: true
}
}
dest_uuid: "f90d94258dce47129eb3ffe48021a93f"
cas_config_opid_index: -1
I20260806 11:36:44.520112 28038 consensus_queue.cc:237] T 00000000000000000000000000000000 P f90d94258dce47129eb3ffe48021a93f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f90d94258dce47129eb3ffe48021a93f" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 46157 } } peers { permanent_uuid: "b160eb10619648eeacfcc3d72bae9fd4" member_type: NON_VOTER last_known_addr { host: "127.21.14.253" port: 43931 } attrs { promote: true } }
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.14.252@KRBTEST.COM" created.
I20260806 11:36:44.522554 28638 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f90d94258dce47129eb3ffe48021a93f [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER b160eb10619648eeacfcc3d72bae9fd4 (127.21.14.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f90d94258dce47129eb3ffe48021a93f" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 46157 } } peers { permanent_uuid: "b160eb10619648eeacfcc3d72bae9fd4" member_type: NON_VOTER last_known_addr { host: "127.21.14.253" port: 43931 } attrs { promote: true } } }
I20260806 11:36:44.523751 28580 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:36:44.525532 28580 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4 [term 1 FOLLOWER]: Refusing update from remote peer f90d94258dce47129eb3ffe48021a93f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260806 11:36:44.526670 28637 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f90d94258dce47129eb3ffe48021a93f [LEADER]: Connected to new peer: Peer: permanent_uuid: "b160eb10619648eeacfcc3d72bae9fd4" member_type: NON_VOTER last_known_addr { host: "127.21.14.253" port: 43931 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 11:36:44.528019 28638 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.21.14.253:43931
I20260806 11:36:44.528597 28638 sys_catalog.cc:455] T 00000000000000000000000000000000 P f90d94258dce47129eb3ffe48021a93f [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "f90d94258dce47129eb3ffe48021a93f" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f90d94258dce47129eb3ffe48021a93f" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 46157 } } peers { permanent_uuid: "b160eb10619648eeacfcc3d72bae9fd4" member_type: NON_VOTER last_known_addr { host: "127.21.14.253" port: 43931 } attrs { promote: true } } }
I20260806 11:36:44.528698 28537 master.cc:537] Master@127.21.14.253:43931 shutting down...
I20260806 11:36:44.528736 28638 sys_catalog.cc:458] T 00000000000000000000000000000000 P f90d94258dce47129eb3ffe48021a93f [sys.catalog]: This master's current role is: LEADER
W20260806 11:36:44.530646 28022 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260806 11:36:44.533849 28022 consensus_peers.cc:487] T 00000000000000000000000000000000 P f90d94258dce47129eb3ffe48021a93f -> Peer b160eb10619648eeacfcc3d72bae9fd4 (127.21.14.253:43931): Couldn't send request to peer b160eb10619648eeacfcc3d72bae9fd4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 11:36:44.533916 28537 raft_consensus.cc:2229] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 11:36:44.534263 28537 raft_consensus.cc:2258] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 11:36:44.534384 28537 tablet_replica.cc:330] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4: stopping tablet replica
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.14.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/krb5kdc/oryx_127.21.14.252.keytab.
Entry for principal oryx/127.21.14.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/krb5kdc/oryx_127.21.14.252.keytab.
I20260806 11:36:44.547586 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.252:40757
--webserver_interface=127.21.14.252
--webserver_port=0
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/krb5kdc/oryx_127.21.14.252.keytab
--principal=oryx/127.21.14.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:43771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.14.254:46157,127.21.14.253:43931,127.21.14.252:40757
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 11:36:44.551381 28537 master.cc:552] Master@127.21.14.253:43931 shutdown complete.
I20260806 11:36:44.551465 28537 master_runner.cc:305] Clearing existing system tablet
I20260806 11:36:44.551777 28537 fs_manager.cc:359] Metadata directory not provided
I20260806 11:36:44.551854 28537 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:36:44.554468 28537 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260806 11:36:44.556210 28647 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:44.556717 28537 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:36:44.556915 28537 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 11:36:44.556973 28537 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/master-1/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/master-1/wal
uuid: "b160eb10619648eeacfcc3d72bae9fd4"
format_stamp: "Formatted at 2026-08-06 11:36:44 on dist-test-slave-7c35"
server_key: "737a6fe322a75a315d0be2e1f364131b"
server_key_iv: "7d7ddd70c4b564586de219491d24d532"
server_key_version: "encryptionkey@0"
I20260806 11:36:44.560463 28537 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 11:36:44.564057 28537 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 11:36:44.564194 28537 log.cc:1196] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4: Deleting WAL directory at /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260806 11:36:44.564486 28537 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4: Deleting consensus metadata
I20260806 11:36:44.564886 28537 master_runner.cc:315] Copying system tablet from 127.21.14.254:46157
I20260806 11:36:44.566484 28537 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4: tablet copy: Beginning tablet copy session from remote peer at address 127.21.14.254:46157
I20260806 11:36:44.577915 28078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:44.567720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40548 (local address 127.21.14.254:46157)
0806 11:36:44.567866 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:44.567871 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:44.567887 (+ 16us) server_negotiation.cc:407] Connection header received
0806 11:36:44.567943 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:44.567945 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:44.567995 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:44.568088 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:44.568996 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:44.569894 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:44.571395 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:44.571779 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:44.571860 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:44.573490 (+ 1630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:44.573495 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:44.573498 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:44.573530 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:44.575428 (+ 1898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:44.576318 (+ 890us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:44.576322 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:44.576324 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:44.576432 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:44.576773 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:44.576776 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:44.576778 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:44.577042 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:44.577121 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:44.577575 (+ 454us) server_negotiation.cc:299] Negotiation successful
0806 11:36:44.577728 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 11:36:44.578882 28654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:44.567277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:46157 (local address 127.0.0.1:40548)
0806 11:36:44.567583 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:44.567601 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:44.567735 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:44.568143 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:44.568146 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:44.568592 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:44.568752 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:44.568763 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:44.570144 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:44.570149 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:44.571210 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:44.571229 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:44.572055 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:44.572058 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:44.572162 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:44.572805 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:44.572821 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:44.573289 (+ 468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:44.575571 (+ 2282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:44.575575 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:44.575577 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:44.575887 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:44.576564 (+ 677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:44.576567 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:44.576570 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:44.576667 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:44.577204 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:44.577206 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:44.577359 (+ 153us) client_negotiation.cc:769] Sending connection context
0806 11:36:44.578444 (+ 1085us) client_negotiation.cc:240] Negotiation successful
0806 11:36:44.578634 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":110,"threads_started":1}
I20260806 11:36:44.579963 28059 tablet_copy_service.cc:140] P f90d94258dce47129eb3ffe48021a93f: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer b160eb10619648eeacfcc3d72bae9fd4 ({username='oryx', principal='oryx/127.21.14.253@KRBTEST.COM'} at 127.0.0.1:40548)
I20260806 11:36:44.580192 28059 tablet_copy_service.cc:161] P f90d94258dce47129eb3ffe48021a93f: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer b160eb10619648eeacfcc3d72bae9fd4 at {username='oryx', principal='oryx/127.21.14.253@KRBTEST.COM'} at 127.0.0.1:40548: session id = b160eb10619648eeacfcc3d72bae9fd4-00000000000000000000000000000000
I20260806 11:36:44.582274 28059 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P f90d94258dce47129eb3ffe48021a93f: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 11:36:44.584628 28537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 11:36:44.587893 28537 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4: tablet copy: Starting download of 0 data blocks...
I20260806 11:36:44.588299 28537 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4: tablet copy: Starting download of 1 WAL segments...
I20260806 11:36:44.591243 28537 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 11:36:44.594165 28059 tablet_copy_service.cc:342] P f90d94258dce47129eb3ffe48021a93f: Request end of tablet copy session b160eb10619648eeacfcc3d72bae9fd4-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.21.14.253@KRBTEST.COM'} at 127.0.0.1:40548
I20260806 11:36:44.594367 28059 tablet_copy_service.cc:433] P f90d94258dce47129eb3ffe48021a93f: ending tablet copy session b160eb10619648eeacfcc3d72bae9fd4-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer b160eb10619648eeacfcc3d72bae9fd4
I20260806 11:36:44.612957 28537 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:36:44.615196 28658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:44.615657 28655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:44.616211 28656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:44.616609 28537 server_base.cc:1034] running on GCE node
I20260806 11:36:44.616845 28537 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:36:44.617219 28537 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:36:44.618414 28537 hybrid_clock.cc:648] HybridClock initialized: now 1786016204618387 us; error 57 us; skew 500 ppm
Aug 06 11:36:44 dist-test-slave-7c35 krb5kdc[27991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016204, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.14.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:36:44.620961 28537 init.cc:377] Logged in from keytab as oryx/127.21.14.253@KRBTEST.COM (short username oryx)
I20260806 11:36:44.621588 28537 webserver.cc:460] Webserver started at http://127.21.14.253:38619/ using document root <none> and password file <none>
I20260806 11:36:44.621805 28537 fs_manager.cc:359] Metadata directory not provided
I20260806 11:36:44.621856 28537 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:36:44.623921 28537 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 11:36:44.625494 28665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:44.625900 28537 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 11:36:44.625999 28537 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/master-1/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/master-1/wal
uuid: "b160eb10619648eeacfcc3d72bae9fd4"
format_stamp: "Formatted at 2026-08-06 11:36:44 on dist-test-slave-7c35"
server_key: "737a6fe322a75a315d0be2e1f364131b"
server_key_iv: "7d7ddd70c4b564586de219491d24d532"
server_key_version: "encryptionkey@0"
I20260806 11:36:44.626111 28537 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:36:44.633617 28239 heartbeater.cc:503] Master 127.21.14.254:46157 was elected leader, sending a full tablet report...
I20260806 11:36:44.648458 28537 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:36:44.651374 28537 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:36:44.656807 28537 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.253:43931
I20260806 11:36:44.657034 28721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.253:43931 every 8 connection(s)
I20260806 11:36:44.657485 28537 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/master-1/data/info.pb
I20260806 11:36:44.659868 28722 sys_catalog.cc:263] Verifying existing consensus state
I20260806 11:36:44.661270 28722 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4: Bootstrap starting.
I20260806 11:36:44.676957 28722 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 11:36:44.677412 28722 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4: Bootstrap complete.
I20260806 11:36:44.678303 28722 raft_consensus.cc:348] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f90d94258dce47129eb3ffe48021a93f" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 46157 } } peers { permanent_uuid: "b160eb10619648eeacfcc3d72bae9fd4" member_type: NON_VOTER last_known_addr { host: "127.21.14.253" port: 43931 } attrs { promote: true } }
I20260806 11:36:44.678478 28722 raft_consensus.cc:729] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b160eb10619648eeacfcc3d72bae9fd4, State: Initialized, Role: LEARNER
I20260806 11:36:44.678604 28722 consensus_queue.cc:260] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f90d94258dce47129eb3ffe48021a93f" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 46157 } } peers { permanent_uuid: "b160eb10619648eeacfcc3d72bae9fd4" member_type: NON_VOTER last_known_addr { host: "127.21.14.253" port: 43931 } attrs { promote: true } }
I20260806 11:36:44.679059 28725 sys_catalog.cc:455] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f90d94258dce47129eb3ffe48021a93f" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 46157 } } peers { permanent_uuid: "b160eb10619648eeacfcc3d72bae9fd4" member_type: NON_VOTER last_known_addr { host: "127.21.14.253" port: 43931 } attrs { promote: true } } }
I20260806 11:36:44.679195 28725 sys_catalog.cc:458] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4 [sys.catalog]: This master's current role is: LEARNER
I20260806 11:36:44.679272 28722 sys_catalog.cc:564] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 11:36:44.683357 28737 catalog_manager.cc:1259] Loaded cluster ID: f64fef229c0a4d3588ddb9a279aad6dd
I20260806 11:36:44.683457 28737 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4: loading cluster ID for follower catalog manager: success
I20260806 11:36:44.685218 28737 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4: acquiring CA information for follower catalog manager: success
I20260806 11:36:44.686539 28737 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 11:36:44.719003 28645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:36:44.719381 28645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:36:44.719439 28645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:36:44.724704 28645 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 11:36:44.724797 28645 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:36:44.724819 28645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:36:44.724838 28645 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 11:36:44.724900 28645 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 11:36:44.731469 28645 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:43771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.21.14.254:46157,127.21.14.253:43931,127.21.14.252:40757
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/krb5kdc/oryx_127.21.14.252.keytab
--principal=oryx/127.21.14.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.252:40757
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.21.14.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28645
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:36:44.732821 28645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:36:44.734035 28645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:36:44.744592 28742 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:44.744529 28746 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:44.744700 28743 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:44.744818 28645 server_base.cc:1034] running on GCE node
I20260806 11:36:44.745736 28645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:36:44.747225 28645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:36:44.748448 28645 hybrid_clock.cc:648] HybridClock initialized: now 1786016204748419 us; error 42 us; skew 500 ppm
Aug 06 11:36:44 dist-test-slave-7c35 krb5kdc[27991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016204, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.14.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:36:44.752087 28645 init.cc:377] Logged in from keytab as oryx/127.21.14.252@KRBTEST.COM (short username oryx)
I20260806 11:36:44.753432 28645 webserver.cc:460] Webserver started at http://127.21.14.252:36437/ using document root <none> and password file <none>
I20260806 11:36:44.754067 28645 fs_manager.cc:359] Metadata directory not provided
I20260806 11:36:44.754144 28645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:36:44.754345 28645 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:36:44.756891 28645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/master-2/data/instance:
uuid: "9e5a72b344fe41a1ae5787da75d1b90f"
format_stamp: "Formatted at 2026-08-06 11:36:44 on dist-test-slave-7c35"
server_key: "8122237d6bb72adba46962c23c278bec"
server_key_iv: "8c6c3fbc0fdeec7596449ec688402b42"
server_key_version: "encryptionkey@0"
I20260806 11:36:44.757642 28645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/master-2/wal/instance:
uuid: "9e5a72b344fe41a1ae5787da75d1b90f"
format_stamp: "Formatted at 2026-08-06 11:36:44 on dist-test-slave-7c35"
server_key: "8122237d6bb72adba46962c23c278bec"
server_key_iv: "8c6c3fbc0fdeec7596449ec688402b42"
server_key_version: "encryptionkey@0"
I20260806 11:36:44.762413 28645 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 11:36:44.765877 28753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:44.767956 28645 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 11:36:44.768195 28645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/master-2/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/master-2/wal
uuid: "9e5a72b344fe41a1ae5787da75d1b90f"
format_stamp: "Formatted at 2026-08-06 11:36:44 on dist-test-slave-7c35"
server_key: "8122237d6bb72adba46962c23c278bec"
server_key_iv: "8c6c3fbc0fdeec7596449ec688402b42"
server_key_version: "encryptionkey@0"
I20260806 11:36:44.768352 28645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:36:44.791224 28645 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:36:44.795724 28645 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:36:44.795926 28645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:36:44.806399 28645 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.252:40757
I20260806 11:36:44.806639 28810 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.252:40757 every 8 connection(s)
I20260806 11:36:44.807932 28645 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/master-2/data/info.pb
I20260806 11:36:44.808301 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 28645
I20260806 11:36:44.812755 28811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:44.817736 28811 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.14.254" port: 46157 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 11:36:44 dist-test-slave-7c35 krb5kdc[27991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016203, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.14.252@KRBTEST.COM
Aug 06 11:36:44 dist-test-slave-7c35 krb5kdc[27991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016204, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.14.252@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM
I20260806 11:36:44.827636 28813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:44.810674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32796 (local address 127.21.14.252:40757)
0806 11:36:44.811746 (+ 1072us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:44.811764 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:44.811809 (+ 45us) server_negotiation.cc:407] Connection header received
0806 11:36:44.812700 (+ 891us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:44.812725 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:44.813198 (+ 473us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:44.813631 (+ 433us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:44.814629 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:44.816068 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:44.817065 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:44.817647 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:44.817859 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:44.821243 (+ 3384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:44.821250 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:44.821267 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:44.821310 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:44.823604 (+ 2294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:44.824289 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:44.824295 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:44.824301 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:44.824467 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:44.824851 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:44.824855 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:44.824857 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:44.825354 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:44.825620 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:44.825961 (+ 341us) server_negotiation.cc:299] Negotiation successful
0806 11:36:44.826309 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":893,"thread_start_us":484,"threads_started":1}
I20260806 11:36:44.833056 28078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:44.820483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40556 (local address 127.21.14.254:46157)
0806 11:36:44.820672 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:44.820678 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:44.821261 (+ 583us) server_negotiation.cc:407] Connection header received
0806 11:36:44.821457 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:44.821462 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:44.821518 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:44.821610 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:44.822492 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:44.823131 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:44.824338 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:44.824614 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:44.824758 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:44.828145 (+ 3387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:44.828148 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:44.828150 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:44.828184 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:44.830426 (+ 2242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:44.831266 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:44.831269 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:44.831271 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:44.831369 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:44.831981 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:44.831984 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:44.831985 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:44.832220 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:44.832321 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:44.832777 (+ 456us) server_negotiation.cc:299] Negotiation successful
0806 11:36:44.832916 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260806 11:36:44.833186 28814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:44.820324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:46157 (local address 127.0.0.1:40556)
0806 11:36:44.821086 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:44.821167 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:44.821336 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:44.821652 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:44.821659 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:44.822192 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:44.822342 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:44.822348 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:44.823278 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:44.823282 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:44.824188 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:44.824203 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:44.824772 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:44.824776 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:44.824970 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:44.825659 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:44.825671 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:44.827823 (+ 2152us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:44.830596 (+ 2773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:44.830604 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:44.830611 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:44.831030 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:44.831599 (+ 569us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:44.831604 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:44.831606 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:44.831835 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:44.832360 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:44.832367 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:44.832516 (+ 149us) client_negotiation.cc:769] Sending connection context
0806 11:36:44.832659 (+ 143us) client_negotiation.cc:240] Negotiation successful
0806 11:36:44.832956 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":346,"threads_started":1}
I20260806 11:36:44.834510 28811 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.14.253" port: 43931 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 11:36:44 dist-test-slave-7c35 krb5kdc[27991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016204, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.14.252@KRBTEST.COM for oryx/127.21.14.253@KRBTEST.COM
I20260806 11:36:44.846900 28814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:44.835156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.253:43931 (local address 127.0.0.1:34214)
0806 11:36:44.835296 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:44.835319 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:44.835396 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:44.835794 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:44.835796 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:44.835966 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:44.836110 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:44.836116 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:44.837802 (+ 1686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:44.837806 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:44.838613 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:44.838628 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:44.839472 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:44.839476 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:44.839585 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:44.840185 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:44.840199 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:44.842021 (+ 1822us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:44.844751 (+ 2730us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:44.844757 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:44.844760 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:44.845180 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:44.845626 (+ 446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:44.845630 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:44.845632 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:44.845743 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:44.846226 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:44.846229 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:44.846345 (+ 116us) client_negotiation.cc:769] Sending connection context
0806 11:36:44.846466 (+ 121us) client_negotiation.cc:240] Negotiation successful
0806 11:36:44.846666 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260806 11:36:44.846900 28815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:44.835287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34214 (local address 127.21.14.253:43931)
0806 11:36:44.835564 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:44.835568 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:44.835581 (+ 13us) server_negotiation.cc:407] Connection header received
0806 11:36:44.835624 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:44.835626 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:44.835668 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:44.835749 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:44.836359 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:44.837407 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:44.838980 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:44.839288 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:44.839399 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:44.842316 (+ 2917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:44.842320 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:44.842322 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:44.842354 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:44.844345 (+ 1991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:44.845391 (+ 1046us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:44.845397 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:44.845398 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:44.845504 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:44.845852 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:44.845856 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:44.845858 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:44.846111 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:44.846235 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:44.846531 (+ 296us) server_negotiation.cc:299] Negotiation successful
0806 11:36:44.846678 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":118,"threads_started":1}
I20260806 11:36:44.848178 28811 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.14.252" port: 40757 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 11:36:44 dist-test-slave-7c35 krb5kdc[27991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016204, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.14.252@KRBTEST.COM for oryx/127.21.14.252@KRBTEST.COM
I20260806 11:36:44.857993 28814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:44.848979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.252:40757 (local address 127.0.0.1:32802)
0806 11:36:44.849128 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:44.849146 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:44.849254 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:44.849505 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:44.849508 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:44.849690 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:44.849800 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:44.849806 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:44.851192 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:44.851195 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:44.851745 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:44.851755 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:44.852319 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:44.852323 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:44.852426 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:44.852989 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:44.853002 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:44.854411 (+ 1409us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:44.856376 (+ 1965us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:44.856381 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:44.856383 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:44.856606 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:44.856906 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:44.856909 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:44.856911 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:44.856995 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:44.857404 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:44.857407 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:44.857495 (+ 88us) client_negotiation.cc:769] Sending connection context
0806 11:36:44.857620 (+ 125us) client_negotiation.cc:240] Negotiation successful
0806 11:36:44.857798 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260806 11:36:44.857992 28813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:44.849019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32802 (local address 127.21.14.252:40757)
0806 11:36:44.849173 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:44.849176 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:44.849198 (+ 22us) server_negotiation.cc:407] Connection header received
0806 11:36:44.849342 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:44.849344 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:44.849384 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:44.849463 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:44.849943 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:44.851022 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:44.851874 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:44.852143 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:44.852237 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:44.854587 (+ 2350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:44.854590 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:44.854592 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:44.854614 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:44.856232 (+ 1618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:44.856714 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:44.856719 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:44.856721 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:44.856802 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:44.857089 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:44.857093 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:44.857095 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:44.857304 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:44.857417 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:44.857678 (+ 261us) server_negotiation.cc:299] Negotiation successful
0806 11:36:44.857861 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":73,"server-negotiator.queue_time_us":62}
I20260806 11:36:44.861953 28811 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f: Bootstrap starting.
I20260806 11:36:44.864691 28811 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:44.865567 28811 log.cc:824] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f: Log is configured to *not* fsync() on all Append() calls
I20260806 11:36:44.868105 28811 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f: No bootstrap required, opened a new log
I20260806 11:36:44.871711 28811 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f90d94258dce47129eb3ffe48021a93f" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 46157 } } peers { permanent_uuid: "b160eb10619648eeacfcc3d72bae9fd4" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 43931 } } peers { permanent_uuid: "9e5a72b344fe41a1ae5787da75d1b90f" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 40757 } }
I20260806 11:36:44.871977 28811 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:44.872048 28811 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e5a72b344fe41a1ae5787da75d1b90f, State: Initialized, Role: FOLLOWER
I20260806 11:36:44.872558 28811 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f90d94258dce47129eb3ffe48021a93f" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 46157 } } peers { permanent_uuid: "b160eb10619648eeacfcc3d72bae9fd4" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 43931 } } peers { permanent_uuid: "9e5a72b344fe41a1ae5787da75d1b90f" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 40757 } }
I20260806 11:36:44.873159 28817 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f90d94258dce47129eb3ffe48021a93f" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 46157 } } peers { permanent_uuid: "b160eb10619648eeacfcc3d72bae9fd4" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 43931 } } peers { permanent_uuid: "9e5a72b344fe41a1ae5787da75d1b90f" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 40757 } } }
I20260806 11:36:44.873411 28817 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 11:36:44.874174 28811 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f [sys.catalog]: configured and running, proceeding with master startup.
W20260806 11:36:44.880039 28829 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 11:36:44.880167 28829 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 11:36:44.882329 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 28101
I20260806 11:36:44.891345 28383 heartbeater.cc:503] Master 127.21.14.254:46157 was elected leader, sending a full tablet report...
I20260806 11:36:44.893599 28645 master_runner.cc:428] Detected that this master 9e5a72b344fe41a1ae5787da75d1b90f is joining an existing cluster
I20260806 11:36:44.893695 28645 master_runner.cc:432] Initiating AddMaster RPC to add 127.21.14.252:40757
I20260806 11:36:44.898298 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.193:38635
--local_ip_for_outbound_sockets=127.21.14.193
--tserver_master_addrs=127.21.14.254:46157
--webserver_port=42231
--webserver_interface=127.21.14.193
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/krb5kdc/oryx_127.21.14.193.keytab
--principal=oryx/127.21.14.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:43771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.21.14.254:46157,127.21.14.253:43931
--tserver_master_addrs=127.21.14.254:46157,127.21.14.253:43931,127.21.14.252:40757 with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 11:36:44.908615 28837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:44.897313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.253:43931 (local address 127.0.0.1:34220)
0806 11:36:44.898572 (+ 1259us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:44.898590 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:44.898786 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:44.899283 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:44.899288 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:44.899587 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:44.899765 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:44.899776 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:44.900828 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:44.900831 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:44.901441 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:44.901448 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:44.902290 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:44.902296 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:44.902421 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:44.903104 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:44.903119 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:44.903692 (+ 573us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:44.906025 (+ 2333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:44.906030 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:44.906033 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:44.906458 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:44.907139 (+ 681us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:44.907143 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:44.907145 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:44.907267 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:44.907880 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:44.907884 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:44.907991 (+ 107us) client_negotiation.cc:769] Sending connection context
0806 11:36:44.908087 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 11:36:44.908259 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1138,"thread_start_us":144,"threads_started":1}
I20260806 11:36:44.909606 28815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:44.898257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34220 (local address 127.21.14.253:43931)
0806 11:36:44.898400 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:44.898405 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:44.898674 (+ 269us) server_negotiation.cc:407] Connection header received
0806 11:36:44.898952 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:44.898958 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:44.899028 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:44.899091 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:44.899945 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:44.900688 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:44.901595 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:44.901985 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:44.902108 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:44.903896 (+ 1788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:44.903902 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:44.903906 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:44.903942 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:44.905863 (+ 1921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:44.906595 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:44.906600 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:44.906602 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:44.906720 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:44.907444 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:44.907448 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:44.907450 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:44.907727 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:44.907830 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:44.909213 (+ 1383us) server_negotiation.cc:299] Negotiation successful
0806 11:36:44.909396 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 11:36:44.913848 28835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:44.897458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:46157 (local address 127.0.0.1:40558)
0806 11:36:44.899112 (+ 1654us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:44.899126 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:44.899288 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:44.899672 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:44.899674 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:44.899905 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:44.900061 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:44.900068 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:44.901201 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:44.901204 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:44.905317 (+ 4113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:44.905333 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:44.906204 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:44.906209 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:44.906335 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:44.906838 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:44.907157 (+ 319us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:44.907699 (+ 542us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:44.911495 (+ 3796us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:44.911499 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:44.911502 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:44.911900 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:44.912416 (+ 516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:44.912420 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:44.912422 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:44.912556 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:44.913188 (+ 632us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:44.913191 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:44.913320 (+ 129us) client_negotiation.cc:769] Sending connection context
0806 11:36:44.913433 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 11:36:44.913625 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1561,"mutex_wait_us":39,"thread_start_us":122,"threads_started":1}
I20260806 11:36:44.914006 28078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:44.898286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40558 (local address 127.21.14.254:46157)
0806 11:36:44.898527 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:44.898531 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:44.899214 (+ 683us) server_negotiation.cc:407] Connection header received
0806 11:36:44.899414 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:44.899417 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:44.899470 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:44.899523 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:44.900216 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:44.901050 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:44.905520 (+ 4470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:44.905937 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:44.906025 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:44.907918 (+ 1893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:44.907925 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:44.907931 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:44.907972 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:44.910530 (+ 2558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:44.912075 (+ 1545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:44.912082 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:44.912085 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:44.912221 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:44.912707 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:44.912714 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:44.912717 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:44.913056 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:44.913185 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:44.913566 (+ 381us) server_negotiation.cc:299] Negotiation successful
0806 11:36:44.913742 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162}
Aug 06 11:36:44 dist-test-slave-7c35 krb5kdc[27991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016203, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.14.254@KRBTEST.COM for oryx/127.21.14.252@KRBTEST.COM
I20260806 11:36:44.935788 28813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:44.898028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32810 (local address 127.21.14.252:40757)
0806 11:36:44.898426 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:44.898431 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:44.901836 (+ 3405us) server_negotiation.cc:407] Connection header received
0806 11:36:44.901960 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:44.901963 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:44.902010 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:44.902086 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:44.902680 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:44.903545 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:44.904710 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:44.905255 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:44.905367 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:44.931700 (+ 26333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:44.931706 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:44.931710 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:44.931741 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:44.933666 (+ 1925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:44.934354 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:44.934358 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:44.934360 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:44.934447 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:44.934744 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:44.934747 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:44.934749 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:44.935022 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:44.935101 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:44.935428 (+ 327us) server_negotiation.cc:299] Negotiation successful
0806 11:36:44.935581 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":115,"server-negotiator.queue_time_us":311,"spinlock_wait_cycles":10624}
I20260806 11:36:44.936667 28836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:44.898139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.252:40757 (local address 127.0.0.1:32810)
0806 11:36:44.901756 (+ 3617us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:44.901771 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:44.901875 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:44.902174 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:44.902177 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:44.902437 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:44.902560 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:44.902567 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:44.903683 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:44.903686 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:44.904584 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:44.904597 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:44.905494 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:44.905497 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:44.905590 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:44.906090 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:44.906107 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:44.931460 (+ 25353us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:44.933836 (+ 2376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:44.933840 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:44.933842 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:44.934240 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:44.934560 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:44.934562 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:44.934564 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:44.934650 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:44.935162 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:44.935165 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:44.935265 (+ 100us) client_negotiation.cc:769] Sending connection context
0806 11:36:44.936302 (+ 1037us) client_negotiation.cc:240] Negotiation successful
0806 11:36:44.936474 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3541,"thread_start_us":107,"threads_started":1}
I20260806 11:36:44.936930 28635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:44.920275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.252:40757 (local address 127.0.0.1:32816)
0806 11:36:44.920498 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:44.920514 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:44.920610 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:44.924887 (+ 4277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:44.924892 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:44.925195 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:44.925350 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:44.925359 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:44.926650 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:44.926656 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:44.927742 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:44.927762 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:44.928615 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:44.928628 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:44.928769 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:44.929462 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:44.929481 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:44.931496 (+ 2015us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:44.934612 (+ 3116us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:44.934617 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:44.934619 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:44.934974 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:44.935389 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:44.935392 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:44.935394 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:44.935497 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:44.935965 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:44.935968 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:44.936282 (+ 314us) client_negotiation.cc:769] Sending connection context
0806 11:36:44.936401 (+ 119us) client_negotiation.cc:240] Negotiation successful
0806 11:36:44.936743 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140}
I20260806 11:36:44.937549 28839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:44.920360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32816 (local address 127.21.14.252:40757)
0806 11:36:44.924352 (+ 3992us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:44.924368 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:44.924394 (+ 26us) server_negotiation.cc:407] Connection header received
0806 11:36:44.924517 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:44.924521 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:44.924598 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:44.924816 (+ 218us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:44.925512 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:44.926448 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:44.927933 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:44.928346 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:44.928478 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:44.931723 (+ 3245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:44.931728 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:44.931730 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:44.931761 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:44.934421 (+ 2660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:44.935136 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:44.935139 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:44.935140 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:44.935262 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:44.935620 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:44.935623 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:44.935625 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:44.935839 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:44.937130 (+ 1291us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:44.937247 (+ 117us) server_negotiation.cc:299] Negotiation successful
0806 11:36:44.937381 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3853,"thread_start_us":147,"threads_started":1}
I20260806 11:36:44.939688 28038 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.21.14.252:40757: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "9e5a72b344fe41a1ae5787da75d1b90f"
member_type: NON_VOTER
last_known_addr {
host: "127.21.14.252"
port: 40757
}
attrs {
promote: true
}
}
dest_uuid: "f90d94258dce47129eb3ffe48021a93f"
cas_config_opid_index: 5
I20260806 11:36:44.940434 28038 consensus_queue.cc:237] T 00000000000000000000000000000000 P f90d94258dce47129eb3ffe48021a93f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f90d94258dce47129eb3ffe48021a93f" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 46157 } } peers { permanent_uuid: "b160eb10619648eeacfcc3d72bae9fd4" member_type: NON_VOTER last_known_addr { host: "127.21.14.253" port: 43931 } attrs { promote: true } } peers { permanent_uuid: "9e5a72b344fe41a1ae5787da75d1b90f" member_type: NON_VOTER last_known_addr { host: "127.21.14.252" port: 40757 } attrs { promote: true } }
I20260806 11:36:44.941960 28642 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f90d94258dce47129eb3ffe48021a93f [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 9e5a72b344fe41a1ae5787da75d1b90f (127.21.14.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f90d94258dce47129eb3ffe48021a93f" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 46157 } } peers { permanent_uuid: "b160eb10619648eeacfcc3d72bae9fd4" member_type: NON_VOTER last_known_addr { host: "127.21.14.253" port: 43931 } attrs { promote: true } } peers { permanent_uuid: "9e5a72b344fe41a1ae5787da75d1b90f" member_type: NON_VOTER last_known_addr { host: "127.21.14.252" port: 40757 } attrs { promote: true } } }
I20260806 11:36:44.943758 28642 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.21.14.252:40757
I20260806 11:36:44.944025 28642 sys_catalog.cc:455] T 00000000000000000000000000000000 P f90d94258dce47129eb3ffe48021a93f [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "f90d94258dce47129eb3ffe48021a93f" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f90d94258dce47129eb3ffe48021a93f" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 46157 } } peers { permanent_uuid: "b160eb10619648eeacfcc3d72bae9fd4" member_type: NON_VOTER last_known_addr { host: "127.21.14.253" port: 43931 } attrs { promote: true } } peers { permanent_uuid: "9e5a72b344fe41a1ae5787da75d1b90f" member_type: NON_VOTER last_known_addr { host: "127.21.14.252" port: 40757 } attrs { promote: true } } }
I20260806 11:36:44.944149 28642 sys_catalog.cc:458] T 00000000000000000000000000000000 P f90d94258dce47129eb3ffe48021a93f [sys.catalog]: This master's current role is: LEADER
I20260806 11:36:44.944648 28645 master.cc:537] Master@127.21.14.252:40757 shutting down...
W20260806 11:36:44.946606 28024 consensus_peers.cc:487] T 00000000000000000000000000000000 P f90d94258dce47129eb3ffe48021a93f -> Peer 9e5a72b344fe41a1ae5787da75d1b90f (127.21.14.252:40757): Couldn't send request to peer 9e5a72b344fe41a1ae5787da75d1b90f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 11:36:44.949743 28645 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f [term 0 FOLLOWER]: Raft consensus shutting down.
I20260806 11:36:44.950158 28645 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f [term 0 FOLLOWER]: Raft consensus is shut down!
I20260806 11:36:44.950318 28645 tablet_replica.cc:330] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f: stopping tablet replica
I20260806 11:36:44.952600 28635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:44.944701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.253:43931 (local address 127.0.0.1:34228)
0806 11:36:44.945432 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:44.945450 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:44.945566 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:44.946253 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:44.946259 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:44.946287 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:36:44.946416 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:44.946427 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:44.949348 (+ 2921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:44.949352 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:44.950914 (+ 1562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:44.950934 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:44.952007 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:44.952011 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:44.952128 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:44.952208 (+ 80us) client_negotiation.cc:769] Sending connection context
0806 11:36:44.952307 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 11:36:44.952424 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":635}
I20260806 11:36:44.952601 28815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:44.944970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34228 (local address 127.21.14.253:43931)
0806 11:36:44.945311 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:44.945318 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:44.945657 (+ 339us) server_negotiation.cc:407] Connection header received
0806 11:36:44.945716 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:44.945719 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:44.945783 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:44.945868 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:36:44.948151 (+ 2283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:44.948987 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:44.951096 (+ 2109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:44.951835 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:44.951980 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:44.952161 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:44.952347 (+ 186us) server_negotiation.cc:299] Negotiation successful
0806 11:36:44.952455 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227}
I20260806 11:36:44.954401 28693 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4 [term 1 LEARNER]: Refusing update from remote peer f90d94258dce47129eb3ffe48021a93f: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 11:36:44.955003 28638 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f90d94258dce47129eb3ffe48021a93f [LEADER]: Connected to new peer: Peer: permanent_uuid: "b160eb10619648eeacfcc3d72bae9fd4" member_type: NON_VOTER last_known_addr { host: "127.21.14.253" port: 43931 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 11:36:44.957574 28693 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 9e5a72b344fe41a1ae5787da75d1b90f (127.21.14.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f90d94258dce47129eb3ffe48021a93f" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 46157 } } peers { permanent_uuid: "b160eb10619648eeacfcc3d72bae9fd4" member_type: NON_VOTER last_known_addr { host: "127.21.14.253" port: 43931 } attrs { promote: true } } peers { permanent_uuid: "9e5a72b344fe41a1ae5787da75d1b90f" member_type: NON_VOTER last_known_addr { host: "127.21.14.252" port: 40757 } attrs { promote: true } } }
I20260806 11:36:44.960476 28725 sys_catalog.cc:455] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f90d94258dce47129eb3ffe48021a93f. Latest consensus state: current_term: 1 leader_uuid: "f90d94258dce47129eb3ffe48021a93f" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f90d94258dce47129eb3ffe48021a93f" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 46157 } } peers { permanent_uuid: "b160eb10619648eeacfcc3d72bae9fd4" member_type: NON_VOTER last_known_addr { host: "127.21.14.253" port: 43931 } attrs { promote: true } } peers { permanent_uuid: "9e5a72b344fe41a1ae5787da75d1b90f" member_type: NON_VOTER last_known_addr { host: "127.21.14.252" port: 40757 } attrs { promote: true } } }
I20260806 11:36:44.960659 28725 sys_catalog.cc:458] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4 [sys.catalog]: This master's current role is: LEARNER
I20260806 11:36:44.960772 28725 sys_catalog.cc:455] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f90d94258dce47129eb3ffe48021a93f" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f90d94258dce47129eb3ffe48021a93f" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 46157 } } peers { permanent_uuid: "b160eb10619648eeacfcc3d72bae9fd4" member_type: NON_VOTER last_known_addr { host: "127.21.14.253" port: 43931 } attrs { promote: true } } peers { permanent_uuid: "9e5a72b344fe41a1ae5787da75d1b90f" member_type: NON_VOTER last_known_addr { host: "127.21.14.252" port: 40757 } attrs { promote: true } } }
I20260806 11:36:44.960845 28725 sys_catalog.cc:458] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4 [sys.catalog]: This master's current role is: LEARNER
I20260806 11:36:44.961887 28638 raft_consensus.cc:1050] T 00000000000000000000000000000000 P f90d94258dce47129eb3ffe48021a93f: attempting to promote NON_VOTER b160eb10619648eeacfcc3d72bae9fd4 to VOTER
I20260806 11:36:44.962461 28638 consensus_queue.cc:237] T 00000000000000000000000000000000 P f90d94258dce47129eb3ffe48021a93f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f90d94258dce47129eb3ffe48021a93f" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 46157 } } peers { permanent_uuid: "b160eb10619648eeacfcc3d72bae9fd4" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 43931 } attrs { promote: false } } peers { permanent_uuid: "9e5a72b344fe41a1ae5787da75d1b90f" member_type: NON_VOTER last_known_addr { host: "127.21.14.252" port: 40757 } attrs { promote: true } }
I20260806 11:36:44.964001 28693 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4 [term 1 LEARNER]: Refusing update from remote peer f90d94258dce47129eb3ffe48021a93f: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260806 11:36:44.964111 28635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:44.963755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.252:40757 (local address 127.0.0.1:32822)
0806 11:36:44.963932 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:44.964012 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.252:40757: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260806 11:36:44.964453 28642 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f90d94258dce47129eb3ffe48021a93f [LEADER]: Connected to new peer: Peer: permanent_uuid: "b160eb10619648eeacfcc3d72bae9fd4" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 43931 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260806 11:36:44.965139 28635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:44.964874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.252:40757 (local address 127.0.0.1:32834)
0806 11:36:44.965019 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:44.965069 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.252:40757: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
W20260806 11:36:44.965328 28024 consensus_peers.cc:487] T 00000000000000000000000000000000 P f90d94258dce47129eb3ffe48021a93f -> Peer 9e5a72b344fe41a1ae5787da75d1b90f (127.21.14.252:40757): Couldn't send request to peer 9e5a72b344fe41a1ae5787da75d1b90f. Status: Network error: Client connection negotiation failed: client connection to 127.21.14.252:40757: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 11:36:44.966492 28641 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f90d94258dce47129eb3ffe48021a93f [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, b160eb10619648eeacfcc3d72bae9fd4 (127.21.14.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f90d94258dce47129eb3ffe48021a93f" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 46157 } } peers { permanent_uuid: "b160eb10619648eeacfcc3d72bae9fd4" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 43931 } attrs { promote: false } } peers { permanent_uuid: "9e5a72b344fe41a1ae5787da75d1b90f" member_type: NON_VOTER last_known_addr { host: "127.21.14.252" port: 40757 } attrs { promote: true } } }
I20260806 11:36:44.967113 28692 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, b160eb10619648eeacfcc3d72bae9fd4 (127.21.14.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f90d94258dce47129eb3ffe48021a93f" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 46157 } } peers { permanent_uuid: "b160eb10619648eeacfcc3d72bae9fd4" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 43931 } attrs { promote: false } } peers { permanent_uuid: "9e5a72b344fe41a1ae5787da75d1b90f" member_type: NON_VOTER last_known_addr { host: "127.21.14.252" port: 40757 } attrs { promote: true } } }
I20260806 11:36:44.968261 28641 sys_catalog.cc:455] T 00000000000000000000000000000000 P f90d94258dce47129eb3ffe48021a93f [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "f90d94258dce47129eb3ffe48021a93f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f90d94258dce47129eb3ffe48021a93f" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 46157 } } peers { permanent_uuid: "b160eb10619648eeacfcc3d72bae9fd4" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 43931 } attrs { promote: false } } peers { permanent_uuid: "9e5a72b344fe41a1ae5787da75d1b90f" member_type: NON_VOTER last_known_addr { host: "127.21.14.252" port: 40757 } attrs { promote: true } } }
I20260806 11:36:44.968372 28641 sys_catalog.cc:458] T 00000000000000000000000000000000 P f90d94258dce47129eb3ffe48021a93f [sys.catalog]: This master's current role is: LEADER
I20260806 11:36:44.968492 28641 sys_catalog.cc:455] T 00000000000000000000000000000000 P f90d94258dce47129eb3ffe48021a93f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f90d94258dce47129eb3ffe48021a93f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f90d94258dce47129eb3ffe48021a93f" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 46157 } } peers { permanent_uuid: "b160eb10619648eeacfcc3d72bae9fd4" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 43931 } attrs { promote: false } } peers { permanent_uuid: "9e5a72b344fe41a1ae5787da75d1b90f" member_type: NON_VOTER last_known_addr { host: "127.21.14.252" port: 40757 } attrs { promote: true } } }
I20260806 11:36:44.968568 28641 sys_catalog.cc:458] T 00000000000000000000000000000000 P f90d94258dce47129eb3ffe48021a93f [sys.catalog]: This master's current role is: LEADER
I20260806 11:36:44.969389 28841 sys_catalog.cc:455] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f90d94258dce47129eb3ffe48021a93f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f90d94258dce47129eb3ffe48021a93f" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 46157 } } peers { permanent_uuid: "b160eb10619648eeacfcc3d72bae9fd4" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 43931 } attrs { promote: false } } peers { permanent_uuid: "9e5a72b344fe41a1ae5787da75d1b90f" member_type: NON_VOTER last_known_addr { host: "127.21.14.252" port: 40757 } attrs { promote: true } } }
I20260806 11:36:44.969534 28841 sys_catalog.cc:458] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 11:36:44.973889 28645 master.cc:552] Master@127.21.14.252:40757 shutdown complete.
I20260806 11:36:44.973965 28645 master_runner.cc:305] Clearing existing system tablet
I20260806 11:36:44.974265 28645 fs_manager.cc:359] Metadata directory not provided
I20260806 11:36:44.974329 28645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:36:44.975932 28645 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 11:36:44.976981 28843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:44.977265 28645 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:36:44.977407 28645 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 11:36:44.977458 28645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/master-2/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/master-2/wal
uuid: "9e5a72b344fe41a1ae5787da75d1b90f"
format_stamp: "Formatted at 2026-08-06 11:36:44 on dist-test-slave-7c35"
server_key: "8122237d6bb72adba46962c23c278bec"
server_key_iv: "8c6c3fbc0fdeec7596449ec688402b42"
server_key_version: "encryptionkey@0"
I20260806 11:36:44.979899 28645 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 11:36:44.982659 28645 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 11:36:44.982785 28645 log.cc:1196] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f: Deleting WAL directory at /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260806 11:36:44.983198 28645 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f: Deleting consensus metadata
I20260806 11:36:44.983385 28645 master_runner.cc:315] Copying system tablet from 127.21.14.254:46157
I20260806 11:36:44.984812 28645 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f: tablet copy: Beginning tablet copy session from remote peer at address 127.21.14.254:46157
I20260806 11:36:44.995617 28078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:44.985603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40566 (local address 127.21.14.254:46157)
0806 11:36:44.985757 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:44.985762 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:44.986051 (+ 289us) server_negotiation.cc:407] Connection header received
0806 11:36:44.986173 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:44.986176 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:44.986237 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:44.986342 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:44.986986 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:44.987900 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:44.989215 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:44.989554 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:44.989676 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:44.991288 (+ 1612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:44.991292 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:44.991295 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:44.991328 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:44.993239 (+ 1911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:44.993999 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:44.994003 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:44.994005 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:44.994124 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:44.994475 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:44.994478 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:44.994480 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:44.994744 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:44.994885 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:44.995300 (+ 415us) server_negotiation.cc:299] Negotiation successful
0806 11:36:44.995444 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 11:36:44.995645 28849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:44.985601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:46157 (local address 127.0.0.1:40566)
0806 11:36:44.985918 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:44.985934 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:44.986074 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:44.986377 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:44.986380 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:44.986670 (+ 290us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:44.986819 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:44.986827 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:44.988081 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:44.988085 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:44.989040 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:44.989055 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:44.989708 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:44.989712 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:44.989820 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:44.990493 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:44.990509 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:44.991095 (+ 586us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:44.993396 (+ 2301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:44.993400 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:44.993402 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:44.993715 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:44.994248 (+ 533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:44.994251 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:44.994253 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:44.994362 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:44.994890 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:44.994894 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:44.995092 (+ 198us) client_negotiation.cc:769] Sending connection context
0806 11:36:44.995220 (+ 128us) client_negotiation.cc:240] Negotiation successful
0806 11:36:44.995403 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":101,"threads_started":1}
I20260806 11:36:44.996532 28059 tablet_copy_service.cc:140] P f90d94258dce47129eb3ffe48021a93f: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 9e5a72b344fe41a1ae5787da75d1b90f ({username='oryx', principal='oryx/127.21.14.252@KRBTEST.COM'} at 127.0.0.1:40566)
I20260806 11:36:44.996655 28059 tablet_copy_service.cc:161] P f90d94258dce47129eb3ffe48021a93f: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 9e5a72b344fe41a1ae5787da75d1b90f at {username='oryx', principal='oryx/127.21.14.252@KRBTEST.COM'} at 127.0.0.1:40566: session id = 9e5a72b344fe41a1ae5787da75d1b90f-00000000000000000000000000000000
I20260806 11:36:44.997884 28059 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P f90d94258dce47129eb3ffe48021a93f: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 11:36:44.999542 28645 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 11:36:45.002466 28645 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f: tablet copy: Starting download of 0 data blocks...
I20260806 11:36:45.002722 28645 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f: tablet copy: Starting download of 1 WAL segments...
I20260806 11:36:45.004827 28645 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 11:36:45.007270 28059 tablet_copy_service.cc:342] P f90d94258dce47129eb3ffe48021a93f: Request end of tablet copy session 9e5a72b344fe41a1ae5787da75d1b90f-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.21.14.252@KRBTEST.COM'} at 127.0.0.1:40566
I20260806 11:36:45.007383 28059 tablet_copy_service.cc:433] P f90d94258dce47129eb3ffe48021a93f: ending tablet copy session 9e5a72b344fe41a1ae5787da75d1b90f-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 9e5a72b344fe41a1ae5787da75d1b90f
I20260806 11:36:45.016861 28645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:36:45.019356 28850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:45.019914 28853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:45.020167 28851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:45.023151 28645 server_base.cc:1034] running on GCE node
I20260806 11:36:45.023416 28645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:36:45.023859 28645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:36:45.025050 28645 hybrid_clock.cc:648] HybridClock initialized: now 1786016205025022 us; error 49 us; skew 500 ppm
Aug 06 11:36:45 dist-test-slave-7c35 krb5kdc[27991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016205, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.14.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:36:45.027254 28645 init.cc:377] Logged in from keytab as oryx/127.21.14.252@KRBTEST.COM (short username oryx)
I20260806 11:36:45.028136 28645 webserver.cc:460] Webserver started at http://127.21.14.252:37827/ using document root <none> and password file <none>
I20260806 11:36:45.028519 28645 fs_manager.cc:359] Metadata directory not provided
I20260806 11:36:45.028592 28645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:36:45.030537 28645 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 11:36:45.032086 28861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:45.032469 28645 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 11:36:45.032577 28645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/master-2/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/master-2/wal
uuid: "9e5a72b344fe41a1ae5787da75d1b90f"
format_stamp: "Formatted at 2026-08-06 11:36:44 on dist-test-slave-7c35"
server_key: "8122237d6bb72adba46962c23c278bec"
server_key_iv: "8c6c3fbc0fdeec7596449ec688402b42"
server_key_version: "encryptionkey@0"
I20260806 11:36:45.032701 28645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:36:45.055581 28645 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:36:45.058504 28645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:36:45.064193 28645 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.252:40757
I20260806 11:36:45.064221 28916 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.252:40757 every 8 connection(s)
I20260806 11:36:45.064891 28645 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/master-2/data/info.pb
I20260806 11:36:45.067279 28918 sys_catalog.cc:263] Verifying existing consensus state
I20260806 11:36:45.068673 28918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f: Bootstrap starting.
I20260806 11:36:45.084427 28918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 11:36:45.084824 28918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f: Bootstrap complete.
I20260806 11:36:45.085494 28918 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f90d94258dce47129eb3ffe48021a93f" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 46157 } } peers { permanent_uuid: "b160eb10619648eeacfcc3d72bae9fd4" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 43931 } attrs { promote: false } } peers { permanent_uuid: "9e5a72b344fe41a1ae5787da75d1b90f" member_type: NON_VOTER last_known_addr { host: "127.21.14.252" port: 40757 } attrs { promote: true } }
I20260806 11:36:45.085664 28918 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 9e5a72b344fe41a1ae5787da75d1b90f, State: Initialized, Role: LEARNER
I20260806 11:36:45.085770 28918 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f90d94258dce47129eb3ffe48021a93f" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 46157 } } peers { permanent_uuid: "b160eb10619648eeacfcc3d72bae9fd4" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 43931 } attrs { promote: false } } peers { permanent_uuid: "9e5a72b344fe41a1ae5787da75d1b90f" member_type: NON_VOTER last_known_addr { host: "127.21.14.252" port: 40757 } attrs { promote: true } }
I20260806 11:36:45.086210 28924 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f90d94258dce47129eb3ffe48021a93f" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 46157 } } peers { permanent_uuid: "b160eb10619648eeacfcc3d72bae9fd4" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 43931 } attrs { promote: false } } peers { permanent_uuid: "9e5a72b344fe41a1ae5787da75d1b90f" member_type: NON_VOTER last_known_addr { host: "127.21.14.252" port: 40757 } attrs { promote: true } } }
I20260806 11:36:45.086339 28918 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 11:36:45.086362 28924 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f [sys.catalog]: This master's current role is: LEARNER
W20260806 11:36:45.087144 28838 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:36:45.087594 28838 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:36:45.087689 28838 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260806 11:36:45.090793 28936 catalog_manager.cc:1259] Loaded cluster ID: f64fef229c0a4d3588ddb9a279aad6dd
I20260806 11:36:45.090941 28936 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f: loading cluster ID for follower catalog manager: success
I20260806 11:36:45.092898 28936 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f: acquiring CA information for follower catalog manager: success
W20260806 11:36:45.093657 28838 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:36:45.093755 28838 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:36:45.093859 28838 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.193
I20260806 11:36:45.094571 28936 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 11:36:45.100255 28838 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:43771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/krb5kdc/oryx_127.21.14.193.keytab
--principal=oryx/127.21.14.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.193:38635
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.14.193
--webserver_port=42231
--tserver_master_addrs=127.21.14.254:46157,127.21.14.253:43931,127.21.14.252:40757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28838
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.193
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:36:45.101617 28838 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:36:45.102918 28838 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:36:45.115760 28938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:45.115840 28939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:45.115841 28941 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:45.116516 28838 server_base.cc:1034] running on GCE node
I20260806 11:36:45.117105 28838 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:36:45.118314 28838 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:36:45.119596 28838 hybrid_clock.cc:648] HybridClock initialized: now 1786016205119496 us; error 114 us; skew 500 ppm
Aug 06 11:36:45 dist-test-slave-7c35 krb5kdc[27991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016205, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.14.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:36:45.123312 28838 init.cc:377] Logged in from keytab as oryx/127.21.14.193@KRBTEST.COM (short username oryx)
I20260806 11:36:45.124905 28838 webserver.cc:460] Webserver started at http://127.21.14.193:42231/ using document root <none> and password file <none>
I20260806 11:36:45.125732 28838 fs_manager.cc:359] Metadata directory not provided
I20260806 11:36:45.125837 28838 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:36:45.131034 28838 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 11:36:45.134171 28948 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:45.136410 28838 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 11:36:45.136656 28838 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-0/wal
uuid: "cb38f9cc24a2413289605d92553b0b60"
format_stamp: "Formatted at 2026-08-06 11:36:43 on dist-test-slave-7c35"
server_key: "507165f35b358b0309cc46aab5f69e0f"
server_key_iv: "22d64c7195f132b1747e50eacbecd45d"
server_key_version: "encryptionkey@0"
I20260806 11:36:45.137390 28838 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:36:45.153260 28532 heartbeater.cc:503] Master 127.21.14.254:46157 was elected leader, sending a full tablet report...
I20260806 11:36:45.155594 28838 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:36:45.160175 28838 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:36:45.160388 28838 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:36:45.161283 28838 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:36:45.162462 28838 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:36:45.162521 28838 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:45.162601 28838 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:36:45.162631 28838 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:45.177554 28838 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.193:38635
I20260806 11:36:45.177667 29069 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.193:38635 every 8 connection(s)
I20260806 11:36:45.178831 28838 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
I20260806 11:36:45.187961 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 28838
I20260806 11:36:45.188263 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 28244
Aug 06 11:36:45 dist-test-slave-7c35 krb5kdc[27991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016205, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.14.193@KRBTEST.COM for oryx/127.21.14.252@KRBTEST.COM
Aug 06 11:36:45 dist-test-slave-7c35 krb5kdc[27991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016205, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.14.193@KRBTEST.COM for oryx/127.21.14.253@KRBTEST.COM
Aug 06 11:36:45 dist-test-slave-7c35 krb5kdc[27991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016205, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.14.193@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM
I20260806 11:36:45.201098 29076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:45.181283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:55873 (local address 127.21.14.252:40757)
0806 11:36:45.181661 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:45.181667 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:45.183347 (+ 1680us) server_negotiation.cc:407] Connection header received
0806 11:36:45.184312 (+ 965us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:45.184316 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:45.184376 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:45.184444 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:45.186667 (+ 2223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.187909 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.190168 (+ 2259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.190518 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.190595 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:45.194875 (+ 4280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:45.194889 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:45.194892 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:45.194932 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:45.197149 (+ 2217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:45.198296 (+ 1147us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:45.198300 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:45.198302 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:45.198439 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:45.199262 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:45.199269 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:45.199273 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:45.199591 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:45.199681 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:45.200632 (+ 951us) server_negotiation.cc:299] Negotiation successful
0806 11:36:45.200816 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":145,"threads_started":1}
I20260806 11:36:45.203478 29078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:45.181873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.252:40757 (local address 127.21.14.193:55873)
0806 11:36:45.183238 (+ 1365us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:45.183254 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:45.183991 (+ 737us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:45.184908 (+ 917us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:45.184921 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:45.185542 (+ 621us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:45.186425 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:45.186441 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.188082 (+ 1641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.188085 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:45.189827 (+ 1742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:45.189849 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.190829 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.190832 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:45.191322 (+ 490us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:45.192237 (+ 915us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:45.192269 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:45.194497 (+ 2228us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:45.197313 (+ 2816us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:45.197322 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:45.197351 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:45.198078 (+ 727us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:45.198784 (+ 706us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:45.198787 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:45.198790 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:45.199119 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:45.199796 (+ 677us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:45.199805 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:45.200268 (+ 463us) client_negotiation.cc:769] Sending connection context
0806 11:36:45.201432 (+ 1164us) client_negotiation.cc:240] Negotiation successful
0806 11:36:45.201961 (+ 529us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1266,"mutex_wait_us":208,"spinlock_wait_cycles":158080,"thread_start_us":194,"threads_started":1}
I20260806 11:36:45.203811 28815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:45.182013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:51917 (local address 127.21.14.253:43931)
0806 11:36:45.182161 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:45.182167 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:45.182819 (+ 652us) server_negotiation.cc:407] Connection header received
0806 11:36:45.184267 (+ 1448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:45.184274 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:45.184341 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:45.184402 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:45.186716 (+ 2314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.187614 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.190342 (+ 2728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.190924 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.191082 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:45.199199 (+ 8117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:45.199207 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:45.199213 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:45.199253 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:45.201377 (+ 2124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:45.202030 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:45.202036 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:45.202041 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:45.202147 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:45.202546 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:45.202552 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:45.202557 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:45.202906 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:45.203004 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:45.203456 (+ 452us) server_negotiation.cc:299] Negotiation successful
0806 11:36:45.203640 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 11:36:45.204725 28078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:45.183715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:33843 (local address 127.21.14.254:46157)
0806 11:36:45.183881 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:45.183886 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:45.183905 (+ 19us) server_negotiation.cc:407] Connection header received
0806 11:36:45.184260 (+ 355us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:45.184265 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:45.184324 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:45.184383 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:45.186666 (+ 2283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.187538 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.196594 (+ 9056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.197028 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.197137 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:45.200655 (+ 3518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:45.200660 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:45.200662 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:45.200696 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:45.202535 (+ 1839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:45.203166 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:45.203170 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:45.203172 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:45.203279 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:45.203637 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:45.203641 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:45.203642 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:45.203884 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:45.203986 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:45.204397 (+ 411us) server_negotiation.cc:299] Negotiation successful
0806 11:36:45.204563 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260806 11:36:45.204993 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.194:36707
--local_ip_for_outbound_sockets=127.21.14.194
--tserver_master_addrs=127.21.14.254:46157
--webserver_port=41511
--webserver_interface=127.21.14.194
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/krb5kdc/oryx_127.21.14.194.keytab
--principal=oryx/127.21.14.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:43771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.21.14.254:46157,127.21.14.253:43931
--tserver_master_addrs=127.21.14.254:46157,127.21.14.253:43931,127.21.14.252:40757 with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 11:36:45.205161 29070 heartbeater.cc:347] Connected to a master server at 127.21.14.252:40757
I20260806 11:36:45.205260 29077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:45.181872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.253:43931 (local address 127.21.14.193:51917)
0806 11:36:45.182592 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:45.182658 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:45.183974 (+ 1316us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:45.184908 (+ 934us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:45.184921 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:45.185517 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:45.186417 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:45.186441 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.187922 (+ 1481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.187926 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:45.190154 (+ 2228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:45.190167 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.196074 (+ 5907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.196078 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:45.196188 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:45.196833 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:45.196855 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:45.198967 (+ 2112us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:45.201570 (+ 2603us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:45.201576 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:45.201581 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:45.201891 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:45.202284 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:45.202290 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:45.202294 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:45.202406 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:45.203099 (+ 693us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:45.203105 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:45.203235 (+ 130us) client_negotiation.cc:769] Sending connection context
0806 11:36:45.204056 (+ 821us) client_negotiation.cc:240] Negotiation successful
0806 11:36:45.204267 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"mutex_wait_us":1021,"spinlock_wait_cycles":37504,"thread_start_us":968,"threads_started":1}
I20260806 11:36:45.205341 29079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:45.183543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:46157 (local address 127.21.14.193:33843)
0806 11:36:45.183830 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:45.183842 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:45.183970 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:45.184976 (+ 1006us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:45.184979 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:45.185511 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:45.186416 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:45.186441 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.194766 (+ 8325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.194773 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:45.195838 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:45.195861 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.197336 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.197340 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:45.197494 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:45.198788 (+ 1294us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:45.198806 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:45.200429 (+ 1623us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:45.202713 (+ 2284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:45.202716 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:45.202718 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:45.203027 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:45.203401 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:45.203404 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:45.203405 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:45.203502 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:45.204052 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:45.204055 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:45.204162 (+ 107us) client_negotiation.cc:769] Sending connection context
0806 11:36:45.204939 (+ 777us) client_negotiation.cc:240] Negotiation successful
0806 11:36:45.205113 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"mutex_wait_us":67,"spinlock_wait_cycles":16384,"thread_start_us":99,"threads_started":1}
I20260806 11:36:45.205385 29070 heartbeater.cc:464] Registering TS with master...
I20260806 11:36:45.206360 29070 heartbeater.cc:511] Master 127.21.14.252:40757 requested a full tablet report, sending...
I20260806 11:36:45.208060 29073 heartbeater.cc:347] Connected to a master server at 127.21.14.253:43931
I20260806 11:36:45.208194 29073 heartbeater.cc:464] Registering TS with master...
I20260806 11:36:45.208456 29073 heartbeater.cc:511] Master 127.21.14.253:43931 requested a full tablet report, sending...
I20260806 11:36:45.208740 29071 heartbeater.cc:347] Connected to a master server at 127.21.14.254:46157
I20260806 11:36:45.208824 29071 heartbeater.cc:464] Registering TS with master...
I20260806 11:36:45.209024 29071 heartbeater.cc:511] Master 127.21.14.254:46157 requested a full tablet report, sending...
I20260806 11:36:45.209087 28879 ts_manager.cc:194] Registered new tserver with Master: cb38f9cc24a2413289605d92553b0b60 (127.21.14.193:38635)
I20260806 11:36:45.210152 28683 ts_manager.cc:194] Registered new tserver with Master: cb38f9cc24a2413289605d92553b0b60 (127.21.14.193:38635)
I20260806 11:36:45.210400 28038 ts_manager.cc:194] Re-registered known tserver with Master: cb38f9cc24a2413289605d92553b0b60 (127.21.14.193:38635)
I20260806 11:36:45.211390 28038 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.14.193@KRBTEST.COM'} at 127.21.14.193:33843
W20260806 11:36:45.343220 29080 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:36:45.343595 29080 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:36:45.343672 29080 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:36:45.349138 29080 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:36:45.349437 29080 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:36:45.349552 29080 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.194
I20260806 11:36:45.356591 29080 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:43771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/krb5kdc/oryx_127.21.14.194.keytab
--principal=oryx/127.21.14.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.194:36707
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.14.194
--webserver_port=41511
--tserver_master_addrs=127.21.14.254:46157,127.21.14.253:43931,127.21.14.252:40757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29080
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.194
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:36:45.357926 29080 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:36:45.359201 29080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:36:45.370563 29085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:45.370563 29089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:45.370750 29080 server_base.cc:1034] running on GCE node
W20260806 11:36:45.371042 29086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:45.371563 29080 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:36:45.372332 29080 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:36:45.373538 29080 hybrid_clock.cc:648] HybridClock initialized: now 1786016205373521 us; error 31 us; skew 500 ppm
Aug 06 11:36:45 dist-test-slave-7c35 krb5kdc[27991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016205, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.14.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:36:45.377269 29080 init.cc:377] Logged in from keytab as oryx/127.21.14.194@KRBTEST.COM (short username oryx)
I20260806 11:36:45.378687 29080 webserver.cc:460] Webserver started at http://127.21.14.194:41511/ using document root <none> and password file <none>
I20260806 11:36:45.379451 29080 fs_manager.cc:359] Metadata directory not provided
I20260806 11:36:45.379539 29080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:36:45.384306 29080 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 11:36:45.387171 29096 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:45.388994 29080 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 11:36:45.389284 29080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-1/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-1/wal
uuid: "0e4d7e5ba5744b47a854f00460dac7aa"
format_stamp: "Formatted at 2026-08-06 11:36:43 on dist-test-slave-7c35"
server_key: "a26c1d5d96ca940da624a6f5dee83708"
server_key_iv: "c03bd8cd911a068dff7e9bc220adef4e"
server_key_version: "encryptionkey@0"
I20260806 11:36:45.390012 29080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:36:45.407048 29080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:36:45.411365 29080 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:36:45.411620 29080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:36:45.412379 29080 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:36:45.413445 29080 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:36:45.413501 29080 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:45.413546 29080 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:36:45.413571 29080 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:45.425684 29080 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.194:36707
I20260806 11:36:45.425704 29217 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.194:36707 every 8 connection(s)
I20260806 11:36:45.427129 29080 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
I20260806 11:36:45.437198 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 29080
I20260806 11:36:45.437394 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 28388
Aug 06 11:36:45 dist-test-slave-7c35 krb5kdc[27991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016205, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.14.194@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM
Aug 06 11:36:45 dist-test-slave-7c35 krb5kdc[27991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016205, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.14.194@KRBTEST.COM for oryx/127.21.14.253@KRBTEST.COM
Aug 06 11:36:45 dist-test-slave-7c35 krb5kdc[27991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016205, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.14.194@KRBTEST.COM for oryx/127.21.14.252@KRBTEST.COM
I20260806 11:36:45.450290 28078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:45.429406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:35009 (local address 127.21.14.254:46157)
0806 11:36:45.429566 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:45.429573 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:45.431251 (+ 1678us) server_negotiation.cc:407] Connection header received
0806 11:36:45.432668 (+ 1417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:45.432673 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:45.432734 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:45.432816 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:45.435691 (+ 2875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.436511 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.438946 (+ 2435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.439226 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.439299 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:45.443000 (+ 3701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:45.443007 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:45.443013 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:45.443052 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:45.445097 (+ 2045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:45.447875 (+ 2778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:45.447882 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:45.447885 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:45.448007 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:45.448582 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:45.448586 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:45.448588 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:45.448886 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:45.448990 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:45.449905 (+ 915us) server_negotiation.cc:299] Negotiation successful
0806 11:36:45.450102 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 11:36:45.451098 29076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:45.431849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:52067 (local address 127.21.14.252:40757)
0806 11:36:45.431981 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:45.431986 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:45.432010 (+ 24us) server_negotiation.cc:407] Connection header received
0806 11:36:45.432543 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:45.432547 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:45.432610 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:45.432688 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:45.434963 (+ 2275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.435483 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.436752 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.437135 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.437215 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:45.447174 (+ 9959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:45.447181 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:45.447188 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:45.447228 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:45.449035 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:45.449548 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:45.449552 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:45.449554 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:45.449651 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:45.449988 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:45.449992 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:45.449993 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:45.450252 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:45.450335 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:45.450742 (+ 407us) server_negotiation.cc:299] Negotiation successful
0806 11:36:45.450925 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 11:36:45.451772 28815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:45.431461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:39087 (local address 127.21.14.253:43931)
0806 11:36:45.431601 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:45.431607 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:45.431627 (+ 20us) server_negotiation.cc:407] Connection header received
0806 11:36:45.432560 (+ 933us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:45.432563 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:45.432625 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:45.432704 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:45.434630 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.435274 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.436628 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.436898 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.437011 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:45.446742 (+ 9731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:45.446749 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:45.446755 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:45.446794 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:45.449246 (+ 2452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:45.449878 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:45.449884 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:45.449889 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:45.450001 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:45.450532 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:45.450539 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:45.450544 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:45.450841 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:45.450957 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:45.451371 (+ 414us) server_negotiation.cc:299] Negotiation successful
0806 11:36:45.451528 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 11:36:45.452298 29224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:45.431540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.252:40757 (local address 127.21.14.194:52067)
0806 11:36:45.431831 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:45.431846 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:45.432308 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:45.433065 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:45.433077 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:45.433657 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:45.434473 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:45.434478 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.435632 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.435639 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:45.436615 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:45.436627 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.437379 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.437382 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:45.444701 (+ 7319us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:45.445250 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:45.445267 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:45.446948 (+ 1681us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:45.449158 (+ 2210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:45.449160 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:45.449162 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:45.449412 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:45.449781 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:45.449785 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:45.449788 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:45.449882 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:45.450422 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:45.450426 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:45.450531 (+ 105us) client_negotiation.cc:769] Sending connection context
0806 11:36:45.451584 (+ 1053us) client_negotiation.cc:240] Negotiation successful
0806 11:36:45.451744 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"mutex_wait_us":145,"spinlock_wait_cycles":20352,"thread_start_us":106,"threads_started":1}
I20260806 11:36:45.452538 29223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:45.430225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.253:43931 (local address 127.21.14.194:39087)
0806 11:36:45.431051 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:45.431127 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:45.432286 (+ 1159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:45.433065 (+ 779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:45.433077 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:45.433655 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:45.434458 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:45.434475 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.435433 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.435436 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:45.436488 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:45.436498 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.437229 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.437235 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:45.442979 (+ 5744us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:45.444043 (+ 1064us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:45.444066 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:45.446499 (+ 2433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:45.449412 (+ 2913us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:45.449418 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:45.449423 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:45.449730 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:45.450290 (+ 560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:45.450296 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:45.450301 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:45.450411 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:45.451049 (+ 638us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:45.451052 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:45.451168 (+ 116us) client_negotiation.cc:769] Sending connection context
0806 11:36:45.452006 (+ 838us) client_negotiation.cc:240] Negotiation successful
0806 11:36:45.452186 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"mutex_wait_us":111,"spinlock_wait_cycles":181888,"thread_start_us":174,"threads_started":1}
I20260806 11:36:45.452626 29222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:45.430225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:46157 (local address 127.21.14.194:35009)
0806 11:36:45.431051 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:45.431127 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:45.432279 (+ 1152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:45.433065 (+ 786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:45.433077 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:45.433655 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:45.434466 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:45.434477 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.436667 (+ 2190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.436673 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:45.438769 (+ 2096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:45.438781 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.439488 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.439491 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:45.439645 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:45.440483 (+ 838us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:45.440505 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:45.442644 (+ 2139us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:45.445302 (+ 2658us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:45.445313 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:45.445342 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:45.447697 (+ 2355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:45.448159 (+ 462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:45.448161 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:45.448162 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:45.448426 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:45.449078 (+ 652us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:45.449089 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:45.449563 (+ 474us) client_negotiation.cc:769] Sending connection context
0806 11:36:45.450602 (+ 1039us) client_negotiation.cc:240] Negotiation successful
0806 11:36:45.451077 (+ 475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"mutex_wait_us":107,"spinlock_wait_cycles":55936,"thread_start_us":185,"threads_started":1}
I20260806 11:36:45.453991 29220 heartbeater.cc:347] Connected to a master server at 127.21.14.253:43931
I20260806 11:36:45.454169 29218 heartbeater.cc:347] Connected to a master server at 127.21.14.252:40757
I20260806 11:36:45.454286 29220 heartbeater.cc:464] Registering TS with master...
I20260806 11:36:45.454285 29218 heartbeater.cc:464] Registering TS with master...
I20260806 11:36:45.454360 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.195:41935
--local_ip_for_outbound_sockets=127.21.14.195
--tserver_master_addrs=127.21.14.254:46157
--webserver_port=42525
--webserver_interface=127.21.14.195
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/krb5kdc/oryx_127.21.14.195.keytab
--principal=oryx/127.21.14.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:43771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.21.14.254:46157,127.21.14.253:43931
--tserver_master_addrs=127.21.14.254:46157,127.21.14.253:43931,127.21.14.252:40757 with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 11:36:45.455073 29218 heartbeater.cc:511] Master 127.21.14.252:40757 requested a full tablet report, sending...
I20260806 11:36:45.455075 29220 heartbeater.cc:511] Master 127.21.14.253:43931 requested a full tablet report, sending...
I20260806 11:36:45.456420 29219 heartbeater.cc:347] Connected to a master server at 127.21.14.254:46157
I20260806 11:36:45.456532 29219 heartbeater.cc:464] Registering TS with master...
I20260806 11:36:45.456624 28683 ts_manager.cc:194] Registered new tserver with Master: 0e4d7e5ba5744b47a854f00460dac7aa (127.21.14.194:36707)
I20260806 11:36:45.456621 28879 ts_manager.cc:194] Registered new tserver with Master: 0e4d7e5ba5744b47a854f00460dac7aa (127.21.14.194:36707)
I20260806 11:36:45.456748 29219 heartbeater.cc:511] Master 127.21.14.254:46157 requested a full tablet report, sending...
I20260806 11:36:45.457588 28038 ts_manager.cc:194] Re-registered known tserver with Master: 0e4d7e5ba5744b47a854f00460dac7aa (127.21.14.194:36707)
I20260806 11:36:45.458485 28038 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.14.194@KRBTEST.COM'} at 127.21.14.194:35009
W20260806 11:36:45.583011 29225 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:36:45.583463 29225 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:36:45.583534 29225 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260806 11:36:45.586001 29076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:45.580093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32836 (local address 127.21.14.252:40757)
0806 11:36:45.580227 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:45.580233 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:45.580492 (+ 259us) server_negotiation.cc:407] Connection header received
0806 11:36:45.580733 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:45.580737 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:45.580804 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:45.580882 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:36:45.581321 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.583221 (+ 1900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.584851 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.585385 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.585499 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:45.585650 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:45.585806 (+ 156us) server_negotiation.cc:299] Negotiation successful
0806 11:36:45.585878 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 11:36:45.586046 29229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:45.579953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.252:40757 (local address 127.0.0.1:32836)
0806 11:36:45.580397 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:45.580413 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:45.580603 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:45.581035 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:45.581038 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:45.581056 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:36:45.581164 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:45.581170 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.583409 (+ 2239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.583417 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:45.584656 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:45.584675 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.585553 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.585557 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:45.585663 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:45.585695 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 11:36:45.585780 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 11:36:45.585878 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":104,"threads_started":1}
I20260806 11:36:45.587131 28890 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260806 11:36:45.587982 28924 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f [sys.catalog]: SysCatalogTable state changed. Reason: New leader f90d94258dce47129eb3ffe48021a93f. Latest consensus state: current_term: 1 leader_uuid: "f90d94258dce47129eb3ffe48021a93f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f90d94258dce47129eb3ffe48021a93f" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 46157 } } peers { permanent_uuid: "b160eb10619648eeacfcc3d72bae9fd4" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 43931 } attrs { promote: false } } peers { permanent_uuid: "9e5a72b344fe41a1ae5787da75d1b90f" member_type: NON_VOTER last_known_addr { host: "127.21.14.252" port: 40757 } attrs { promote: true } } }
I20260806 11:36:45.588184 28924 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f [sys.catalog]: This master's current role is: LEARNER
W20260806 11:36:45.589506 29225 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:36:45.589608 29225 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:36:45.589721 29225 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.195
I20260806 11:36:45.596040 29225 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:43771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/krb5kdc/oryx_127.21.14.195.keytab
--principal=oryx/127.21.14.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.195:41935
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.14.195
--webserver_port=42525
--tserver_master_addrs=127.21.14.254:46157,127.21.14.253:43931,127.21.14.252:40757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29225
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.195
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:36:45.597378 29225 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:36:45.598587 29225 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:36:45.610096 29231 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:45.610066 29234 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:45.610066 29232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:45.610443 29225 server_base.cc:1034] running on GCE node
I20260806 11:36:45.611095 29225 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:36:45.612069 29225 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:36:45.613289 29225 hybrid_clock.cc:648] HybridClock initialized: now 1786016205613281 us; error 93 us; skew 500 ppm
Aug 06 11:36:45 dist-test-slave-7c35 krb5kdc[27991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016205, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.14.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:36:45.617980 29225 init.cc:377] Logged in from keytab as oryx/127.21.14.195@KRBTEST.COM (short username oryx)
I20260806 11:36:45.619544 29225 webserver.cc:460] Webserver started at http://127.21.14.195:42525/ using document root <none> and password file <none>
I20260806 11:36:45.620512 29225 fs_manager.cc:359] Metadata directory not provided
I20260806 11:36:45.620641 29225 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:36:45.626407 29225 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 11:36:45.629226 29241 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:45.631083 29225 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 11:36:45.631269 29225 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-2/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-2/wal
uuid: "98033857a04c4ac794f0feeead2ddc3d"
format_stamp: "Formatted at 2026-08-06 11:36:44 on dist-test-slave-7c35"
server_key: "fbe86ef7178c7583cb7bf7e3a5eed884"
server_key_iv: "f317588ef27887ad0b713f9a7fe4bc71"
server_key_version: "encryptionkey@0"
I20260806 11:36:45.631953 29225 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:36:45.648304 29225 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:36:45.652733 29225 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:36:45.652987 29225 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:36:45.653712 29225 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:36:45.654839 29225 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:36:45.654965 29225 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:45.655041 29225 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:36:45.655073 29225 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:45.666582 29225 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.195:41935
I20260806 11:36:45.666632 29358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.195:41935 every 8 connection(s)
I20260806 11:36:45.667896 29225 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
I20260806 11:36:45.674494 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 29225
Aug 06 11:36:45 dist-test-slave-7c35 krb5kdc[27991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016205, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.14.195@KRBTEST.COM for oryx/127.21.14.252@KRBTEST.COM
Aug 06 11:36:45 dist-test-slave-7c35 krb5kdc[27991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016205, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.14.195@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM
I20260806 11:36:45.688364 29076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:45.670211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:40785 (local address 127.21.14.252:40757)
0806 11:36:45.670351 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:45.670363 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:45.671973 (+ 1610us) server_negotiation.cc:407] Connection header received
0806 11:36:45.673134 (+ 1161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:45.673138 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:45.673206 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:45.673318 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:45.675360 (+ 2042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.676192 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.678446 (+ 2254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.678806 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.678934 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:45.683029 (+ 4095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:45.683034 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:45.683038 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:45.683069 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:45.685064 (+ 1995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:45.685848 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:45.685854 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:45.685856 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:45.685956 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:45.686486 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:45.686491 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:45.686493 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:45.686777 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:45.686899 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:45.688016 (+ 1117us) server_negotiation.cc:299] Negotiation successful
0806 11:36:45.688167 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 11:36:45.689539 28078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:45.672133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:53091 (local address 127.21.14.254:46157)
0806 11:36:45.672273 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:45.672278 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:45.672553 (+ 275us) server_negotiation.cc:407] Connection header received
0806 11:36:45.673281 (+ 728us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:45.673284 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:45.673333 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:45.673427 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:45.675359 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.676218 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.678439 (+ 2221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.678806 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.678952 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:45.683614 (+ 4662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:45.683619 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:45.683624 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:45.683670 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:45.686007 (+ 2337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:45.686552 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:45.686555 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:45.686557 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:45.686645 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:45.686999 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:45.687002 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:45.687004 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:45.687734 (+ 730us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:45.687836 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:45.689258 (+ 1422us) server_negotiation.cc:299] Negotiation successful
0806 11:36:45.689391 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 11:36:45.690469 29366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:45.672028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:46157 (local address 127.21.14.195:53091)
0806 11:36:45.672432 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:45.672449 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:45.672860 (+ 411us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:45.673532 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:45.673538 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:45.674118 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:45.675159 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:45.675183 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.676428 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.676432 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:45.678283 (+ 1851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:45.678289 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.679139 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.679143 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:45.679499 (+ 356us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:45.680450 (+ 951us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:45.680479 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:45.683409 (+ 2930us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:45.686179 (+ 2770us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:45.686181 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:45.686183 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:45.686432 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:45.686759 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:45.686762 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:45.686764 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:45.686875 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:45.687883 (+ 1008us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:45.687889 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:45.688231 (+ 342us) client_negotiation.cc:769] Sending connection context
0806 11:36:45.688369 (+ 138us) client_negotiation.cc:240] Negotiation successful
0806 11:36:45.688804 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"mutex_wait_us":571,"spinlock_wait_cycles":41216,"thread_start_us":94,"threads_started":1}
I20260806 11:36:45.691030 29364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:45.670736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.252:40757 (local address 127.21.14.195:40785)
0806 11:36:45.671856 (+ 1120us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:45.671871 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:45.672875 (+ 1004us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:45.673528 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:45.673536 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:45.674146 (+ 610us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:45.675159 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:45.675184 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.676429 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.676443 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:45.678268 (+ 1825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:45.678285 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.679139 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.679143 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:45.679503 (+ 360us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:45.680761 (+ 1258us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:45.680781 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:45.682683 (+ 1902us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:45.685225 (+ 2542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:45.685234 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:45.685259 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:45.685712 (+ 453us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:45.686083 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:45.686087 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:45.686089 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:45.686313 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:45.687227 (+ 914us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:45.687235 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:45.687704 (+ 469us) client_negotiation.cc:769] Sending connection context
0806 11:36:45.688867 (+ 1163us) client_negotiation.cc:240] Negotiation successful
0806 11:36:45.689013 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1014,"mutex_wait_us":628,"spinlock_wait_cycles":6784,"thread_start_us":221,"threads_started":1}
I20260806 11:36:45.691311 29374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:45.678155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32844 (local address 127.21.14.252:40757)
0806 11:36:45.679401 (+ 1246us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:45.679405 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:45.680897 (+ 1492us) server_negotiation.cc:407] Connection header received
0806 11:36:45.680949 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:45.680952 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:45.681007 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:45.681124 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:45.682067 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.682890 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.684148 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.684549 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.684669 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:45.686905 (+ 2236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:45.686909 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:45.686912 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:45.686937 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:45.688969 (+ 2032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:45.689422 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:45.689429 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:45.689431 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:45.689527 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:45.689876 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:45.689883 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:45.689886 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:45.690168 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:45.690314 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:45.690998 (+ 684us) server_negotiation.cc:299] Negotiation successful
0806 11:36:45.691144 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1161,"thread_start_us":110,"threads_started":1}
Aug 06 11:36:45 dist-test-slave-7c35 krb5kdc[27991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016205, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.14.195@KRBTEST.COM for oryx/127.21.14.253@KRBTEST.COM
I20260806 11:36:45.691890 29360 heartbeater.cc:347] Connected to a master server at 127.21.14.254:46157
I20260806 11:36:45.692174 29360 heartbeater.cc:464] Registering TS with master...
I20260806 11:36:45.692849 29360 heartbeater.cc:511] Master 127.21.14.254:46157 requested a full tablet report, sending...
I20260806 11:36:45.693387 29359 heartbeater.cc:347] Connected to a master server at 127.21.14.252:40757
I20260806 11:36:45.693476 29359 heartbeater.cc:464] Registering TS with master...
I20260806 11:36:45.693655 29359 heartbeater.cc:511] Master 127.21.14.252:40757 requested a full tablet report, sending...
I20260806 11:36:45.694125 28038 ts_manager.cc:194] Re-registered known tserver with Master: 98033857a04c4ac794f0feeead2ddc3d (127.21.14.195:41935)
I20260806 11:36:45.694793 28038 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.14.195@KRBTEST.COM'} at 127.21.14.195:53091
I20260806 11:36:45.695669 28879 ts_manager.cc:194] Registered new tserver with Master: 98033857a04c4ac794f0feeead2ddc3d (127.21.14.195:41935)
I20260806 11:36:45.696329 28815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:45.671004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:52353 (local address 127.21.14.253:43931)
0806 11:36:45.671166 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:45.671170 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:45.671750 (+ 580us) server_negotiation.cc:407] Connection header received
0806 11:36:45.673558 (+ 1808us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:45.673562 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:45.673612 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:45.673702 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:45.675359 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.676218 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.678445 (+ 2227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.678791 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.678919 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:45.692391 (+ 13472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:45.692396 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:45.692399 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:45.692428 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:45.694239 (+ 1811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:45.694800 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:45.694804 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:45.694806 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:45.694979 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:45.695307 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:45.695310 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:45.695312 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:45.695564 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:45.695672 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:45.696024 (+ 352us) server_negotiation.cc:299] Negotiation successful
0806 11:36:45.696155 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 11:36:45.697022 29363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:45.670736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.253:43931 (local address 127.21.14.195:52353)
0806 11:36:45.671548 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:45.671625 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:45.672860 (+ 1235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:45.673751 (+ 891us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:45.673757 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:45.674149 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:45.675163 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:45.675183 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.676428 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.676432 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:45.678282 (+ 1850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:45.678297 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.679147 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.679152 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:45.679531 (+ 379us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:45.680992 (+ 1461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:45.681007 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:45.692177 (+ 11170us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:45.694374 (+ 2197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:45.694377 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:45.694380 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:45.694651 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:45.695106 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:45.695109 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:45.695111 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:45.695206 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:45.695689 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:45.695693 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:45.695801 (+ 108us) client_negotiation.cc:769] Sending connection context
0806 11:36:45.696556 (+ 755us) client_negotiation.cc:240] Negotiation successful
0806 11:36:45.696729 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"mutex_wait_us":530,"spinlock_wait_cycles":29184,"thread_start_us":753,"threads_started":1}
I20260806 11:36:45.698195 29361 heartbeater.cc:347] Connected to a master server at 127.21.14.253:43931
I20260806 11:36:45.698309 29361 heartbeater.cc:464] Registering TS with master...
I20260806 11:36:45.698562 29361 heartbeater.cc:511] Master 127.21.14.253:43931 requested a full tablet report, sending...
I20260806 11:36:45.699698 28683 ts_manager.cc:194] Registered new tserver with Master: 98033857a04c4ac794f0feeead2ddc3d (127.21.14.195:41935)
I20260806 11:36:45.699932 29371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:45.677154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34240 (local address 127.21.14.253:43931)
0806 11:36:45.679098 (+ 1944us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:45.679102 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:45.679866 (+ 764us) server_negotiation.cc:407] Connection header received
0806 11:36:45.679971 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:45.679974 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:45.680029 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:45.680114 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:45.681271 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.682148 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.683354 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.686604 (+ 3250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.686731 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:45.688621 (+ 1890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:45.688625 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:45.688630 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:45.688662 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:45.691458 (+ 2796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:45.698234 (+ 6776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:45.698238 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:45.698240 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:45.698348 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:45.698674 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:45.698681 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:45.698683 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:45.698982 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:45.699124 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:45.699520 (+ 396us) server_negotiation.cc:299] Negotiation successful
0806 11:36:45.699725 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1858,"thread_start_us":70,"threads_started":1}
I20260806 11:36:45.702345 29373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:45.677689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40582 (local address 127.21.14.254:46157)
0806 11:36:45.680130 (+ 2441us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:45.680144 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:45.680158 (+ 14us) server_negotiation.cc:407] Connection header received
0806 11:36:45.680200 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:45.680204 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:45.680253 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:45.680348 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:45.693689 (+ 13341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.694631 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.695892 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.696260 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.696349 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:45.698242 (+ 1893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:45.698246 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:45.698248 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:45.698281 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:45.700510 (+ 2229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:45.701006 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:45.701010 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:45.701011 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:45.701109 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:45.701403 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:45.701406 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:45.701408 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:45.701619 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:45.701769 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:45.702011 (+ 242us) server_negotiation.cc:299] Negotiation successful
0806 11:36:45.702152 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2347,"thread_start_us":75,"threads_started":1}
I20260806 11:36:45.705714 28038 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40582:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 11:36:45.709309 28038 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 11:36:45.712863 28638 sys_catalog.cc:455] T 00000000000000000000000000000000 P f90d94258dce47129eb3ffe48021a93f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f90d94258dce47129eb3ffe48021a93f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f90d94258dce47129eb3ffe48021a93f" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 46157 } } peers { permanent_uuid: "b160eb10619648eeacfcc3d72bae9fd4" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 43931 } attrs { promote: false } } peers { permanent_uuid: "9e5a72b344fe41a1ae5787da75d1b90f" member_type: NON_VOTER last_known_addr { host: "127.21.14.252" port: 40757 } attrs { promote: true } } }
I20260806 11:36:45.713080 28638 sys_catalog.cc:458] T 00000000000000000000000000000000 P f90d94258dce47129eb3ffe48021a93f [sys.catalog]: This master's current role is: LEADER
I20260806 11:36:45.717017 28638 raft_consensus.cc:1050] T 00000000000000000000000000000000 P f90d94258dce47129eb3ffe48021a93f: attempting to promote NON_VOTER 9e5a72b344fe41a1ae5787da75d1b90f to VOTER
I20260806 11:36:45.717375 28638 consensus_queue.cc:237] T 00000000000000000000000000000000 P f90d94258dce47129eb3ffe48021a93f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f90d94258dce47129eb3ffe48021a93f" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 46157 } } peers { permanent_uuid: "b160eb10619648eeacfcc3d72bae9fd4" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 43931 } attrs { promote: false } } peers { permanent_uuid: "9e5a72b344fe41a1ae5787da75d1b90f" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 40757 } attrs { promote: false } }
I20260806 11:36:45.718765 28890 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f [term 1 LEARNER]: Refusing update from remote peer f90d94258dce47129eb3ffe48021a93f: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260806 11:36:45.719256 29380 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f90d94258dce47129eb3ffe48021a93f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e5a72b344fe41a1ae5787da75d1b90f" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 40757 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260806 11:36:45.721319 28692 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4 [term 1 FOLLOWER]: Refusing update from remote peer f90d94258dce47129eb3ffe48021a93f: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260806 11:36:45.721601 29379 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f90d94258dce47129eb3ffe48021a93f [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 9e5a72b344fe41a1ae5787da75d1b90f (127.21.14.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f90d94258dce47129eb3ffe48021a93f" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 46157 } } peers { permanent_uuid: "b160eb10619648eeacfcc3d72bae9fd4" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 43931 } attrs { promote: false } } peers { permanent_uuid: "9e5a72b344fe41a1ae5787da75d1b90f" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 40757 } attrs { promote: false } } }
I20260806 11:36:45.721804 29380 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f90d94258dce47129eb3ffe48021a93f [LEADER]: Connected to new peer: Peer: permanent_uuid: "b160eb10619648eeacfcc3d72bae9fd4" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 43931 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260806 11:36:45.721567 28890 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 9e5a72b344fe41a1ae5787da75d1b90f (127.21.14.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f90d94258dce47129eb3ffe48021a93f" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 46157 } } peers { permanent_uuid: "b160eb10619648eeacfcc3d72bae9fd4" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 43931 } attrs { promote: false } } peers { permanent_uuid: "9e5a72b344fe41a1ae5787da75d1b90f" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 40757 } attrs { promote: false } } }
I20260806 11:36:45.723359 29379 sys_catalog.cc:455] T 00000000000000000000000000000000 P f90d94258dce47129eb3ffe48021a93f [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "f90d94258dce47129eb3ffe48021a93f" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f90d94258dce47129eb3ffe48021a93f" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 46157 } } peers { permanent_uuid: "b160eb10619648eeacfcc3d72bae9fd4" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 43931 } attrs { promote: false } } peers { permanent_uuid: "9e5a72b344fe41a1ae5787da75d1b90f" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 40757 } attrs { promote: false } } }
I20260806 11:36:45.723239 28692 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 9e5a72b344fe41a1ae5787da75d1b90f (127.21.14.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f90d94258dce47129eb3ffe48021a93f" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 46157 } } peers { permanent_uuid: "b160eb10619648eeacfcc3d72bae9fd4" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 43931 } attrs { promote: false } } peers { permanent_uuid: "9e5a72b344fe41a1ae5787da75d1b90f" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 40757 } attrs { promote: false } } }
I20260806 11:36:45.723469 29379 sys_catalog.cc:458] T 00000000000000000000000000000000 P f90d94258dce47129eb3ffe48021a93f [sys.catalog]: This master's current role is: LEADER
I20260806 11:36:45.723594 29379 sys_catalog.cc:455] T 00000000000000000000000000000000 P f90d94258dce47129eb3ffe48021a93f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f90d94258dce47129eb3ffe48021a93f" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f90d94258dce47129eb3ffe48021a93f" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 46157 } } peers { permanent_uuid: "b160eb10619648eeacfcc3d72bae9fd4" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 43931 } attrs { promote: false } } peers { permanent_uuid: "9e5a72b344fe41a1ae5787da75d1b90f" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 40757 } attrs { promote: false } } }
I20260806 11:36:45.723681 29379 sys_catalog.cc:458] T 00000000000000000000000000000000 P f90d94258dce47129eb3ffe48021a93f [sys.catalog]: This master's current role is: LEADER
I20260806 11:36:45.723634 28924 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f90d94258dce47129eb3ffe48021a93f" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f90d94258dce47129eb3ffe48021a93f" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 46157 } } peers { permanent_uuid: "b160eb10619648eeacfcc3d72bae9fd4" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 43931 } attrs { promote: false } } peers { permanent_uuid: "9e5a72b344fe41a1ae5787da75d1b90f" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 40757 } attrs { promote: false } } }
I20260806 11:36:45.723754 28924 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 11:36:45.726290 29379 sys_catalog.cc:455] T 00000000000000000000000000000000 P f90d94258dce47129eb3ffe48021a93f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f90d94258dce47129eb3ffe48021a93f" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f90d94258dce47129eb3ffe48021a93f" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 46157 } } peers { permanent_uuid: "b160eb10619648eeacfcc3d72bae9fd4" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 43931 } attrs { promote: false } } peers { permanent_uuid: "9e5a72b344fe41a1ae5787da75d1b90f" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 40757 } attrs { promote: false } } }
I20260806 11:36:45.726450 29379 sys_catalog.cc:458] T 00000000000000000000000000000000 P f90d94258dce47129eb3ffe48021a93f [sys.catalog]: This master's current role is: LEADER
I20260806 11:36:45.728080 29388 sys_catalog.cc:455] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f90d94258dce47129eb3ffe48021a93f" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f90d94258dce47129eb3ffe48021a93f" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 46157 } } peers { permanent_uuid: "b160eb10619648eeacfcc3d72bae9fd4" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 43931 } attrs { promote: false } } peers { permanent_uuid: "9e5a72b344fe41a1ae5787da75d1b90f" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 40757 } attrs { promote: false } } }
I20260806 11:36:45.728331 29388 sys_catalog.cc:458] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 11:36:45.735203 29390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:45.727971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.193:38635 (local address 127.0.0.1:44350)
0806 11:36:45.728631 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:45.728646 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:45.728736 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:45.730120 (+ 1384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:45.730124 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:45.730142 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:36:45.730253 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:45.730261 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.731812 (+ 1551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.731816 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:45.733081 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:45.733092 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.734573 (+ 1481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.734577 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:45.734740 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:45.734762 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 11:36:45.734910 (+ 148us) client_negotiation.cc:240] Negotiation successful
0806 11:36:45.735010 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"mutex_wait_us":77,"thread_start_us":115,"threads_started":1}
I20260806 11:36:45.735293 29391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:45.728224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44350 (local address 127.21.14.193:38635)
0806 11:36:45.729226 (+ 1002us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:45.729237 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:45.729312 (+ 75us) server_negotiation.cc:407] Connection header received
0806 11:36:45.729413 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:45.729420 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:45.729737 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:45.729917 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:36:45.730450 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.731680 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.733227 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.734211 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.734318 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:45.734504 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:45.734930 (+ 426us) server_negotiation.cc:299] Negotiation successful
0806 11:36:45.735048 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":852,"thread_start_us":170,"threads_started":1}
I20260806 11:36:45.735626 29229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:45.727702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.195:41935 (local address 127.0.0.1:33782)
0806 11:36:45.728185 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:45.728201 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:45.728298 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:45.731097 (+ 2799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:45.731100 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:45.731119 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:36:45.731202 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:45.731208 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.732801 (+ 1593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.732804 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:45.733984 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:45.733995 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.735202 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.735206 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:45.735304 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:45.735328 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 11:36:45.735385 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 11:36:45.735483 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":396,"mutex_wait_us":1}
I20260806 11:36:45.736122 29394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:45.729555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34088 (local address 127.21.14.194:36707)
0806 11:36:45.729919 (+ 364us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:45.729932 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:45.729958 (+ 26us) server_negotiation.cc:407] Connection header received
0806 11:36:45.730037 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:45.730042 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:45.730249 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:45.730414 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:36:45.731132 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.732219 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.733463 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.734320 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.734425 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:45.734586 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:45.735828 (+ 1242us) server_negotiation.cc:299] Negotiation successful
0806 11:36:45.735960 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":120,"threads_started":1}
I20260806 11:36:45.735245 29392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:45.729393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.194:36707 (local address 127.0.0.1:34088)
0806 11:36:45.729692 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:45.729707 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:45.729807 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:45.730415 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:45.730418 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:45.730441 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:36:45.730980 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:45.730989 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.732389 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.732392 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:45.733313 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:45.733324 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.734654 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.734658 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:45.734767 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:45.734792 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 11:36:45.734948 (+ 156us) client_negotiation.cc:240] Negotiation successful
0806 11:36:45.735038 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"mutex_wait_us":3,"thread_start_us":130,"threads_started":1}
I20260806 11:36:45.736389 29389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:45.727764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33782 (local address 127.21.14.195:41935)
0806 11:36:45.730588 (+ 2824us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:45.730600 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:45.730630 (+ 30us) server_negotiation.cc:407] Connection header received
0806 11:36:45.730689 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:45.730693 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:45.730905 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:45.731365 (+ 460us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:36:45.731527 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.732662 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.734190 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.735065 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.735845 (+ 780us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:45.736016 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:45.736069 (+ 53us) server_negotiation.cc:299] Negotiation successful
0806 11:36:45.736232 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2704,"thread_start_us":128,"threads_started":1}
I20260806 11:36:45.739375 29291 tablet_service.cc:1467] Processing CreateTablet for tablet 27f463bfebf745569c3e2a9e9ae37bfd (DEFAULT_TABLE table=test-table [id=a3e9a7a15bdc4e91a53cb7fb8addb092]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:36:45.739197 28997 tablet_service.cc:1467] Processing CreateTablet for tablet 27f463bfebf745569c3e2a9e9ae37bfd (DEFAULT_TABLE table=test-table [id=a3e9a7a15bdc4e91a53cb7fb8addb092]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:36:45.739463 29145 tablet_service.cc:1467] Processing CreateTablet for tablet 27f463bfebf745569c3e2a9e9ae37bfd (DEFAULT_TABLE table=test-table [id=a3e9a7a15bdc4e91a53cb7fb8addb092]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:36:45.740481 29291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27f463bfebf745569c3e2a9e9ae37bfd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:45.740518 29145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27f463bfebf745569c3e2a9e9ae37bfd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:45.740518 28997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27f463bfebf745569c3e2a9e9ae37bfd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:45.749212 29395 tablet_bootstrap.cc:492] T 27f463bfebf745569c3e2a9e9ae37bfd P 98033857a04c4ac794f0feeead2ddc3d: Bootstrap starting.
I20260806 11:36:45.750633 29396 tablet_bootstrap.cc:492] T 27f463bfebf745569c3e2a9e9ae37bfd P cb38f9cc24a2413289605d92553b0b60: Bootstrap starting.
I20260806 11:36:45.751248 29397 tablet_bootstrap.cc:492] T 27f463bfebf745569c3e2a9e9ae37bfd P 0e4d7e5ba5744b47a854f00460dac7aa: Bootstrap starting.
I20260806 11:36:45.751724 29395 tablet_bootstrap.cc:654] T 27f463bfebf745569c3e2a9e9ae37bfd P 98033857a04c4ac794f0feeead2ddc3d: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:45.752525 29395 log.cc:824] T 27f463bfebf745569c3e2a9e9ae37bfd P 98033857a04c4ac794f0feeead2ddc3d: Log is configured to *not* fsync() on all Append() calls
I20260806 11:36:45.753361 29396 tablet_bootstrap.cc:654] T 27f463bfebf745569c3e2a9e9ae37bfd P cb38f9cc24a2413289605d92553b0b60: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:45.753379 29397 tablet_bootstrap.cc:654] T 27f463bfebf745569c3e2a9e9ae37bfd P 0e4d7e5ba5744b47a854f00460dac7aa: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:45.754280 29397 log.cc:824] T 27f463bfebf745569c3e2a9e9ae37bfd P 0e4d7e5ba5744b47a854f00460dac7aa: Log is configured to *not* fsync() on all Append() calls
I20260806 11:36:45.754281 29396 log.cc:824] T 27f463bfebf745569c3e2a9e9ae37bfd P cb38f9cc24a2413289605d92553b0b60: Log is configured to *not* fsync() on all Append() calls
I20260806 11:36:45.755075 29395 tablet_bootstrap.cc:492] T 27f463bfebf745569c3e2a9e9ae37bfd P 98033857a04c4ac794f0feeead2ddc3d: No bootstrap required, opened a new log
I20260806 11:36:45.755306 29395 ts_tablet_manager.cc:1397] T 27f463bfebf745569c3e2a9e9ae37bfd P 98033857a04c4ac794f0feeead2ddc3d: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 11:36:45.756397 29396 tablet_bootstrap.cc:492] T 27f463bfebf745569c3e2a9e9ae37bfd P cb38f9cc24a2413289605d92553b0b60: No bootstrap required, opened a new log
I20260806 11:36:45.756633 29396 ts_tablet_manager.cc:1397] T 27f463bfebf745569c3e2a9e9ae37bfd P cb38f9cc24a2413289605d92553b0b60: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 11:36:45.756711 29397 tablet_bootstrap.cc:492] T 27f463bfebf745569c3e2a9e9ae37bfd P 0e4d7e5ba5744b47a854f00460dac7aa: No bootstrap required, opened a new log
I20260806 11:36:45.756906 29397 ts_tablet_manager.cc:1397] T 27f463bfebf745569c3e2a9e9ae37bfd P 0e4d7e5ba5744b47a854f00460dac7aa: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 11:36:45.759949 29396 raft_consensus.cc:348] T 27f463bfebf745569c3e2a9e9ae37bfd P cb38f9cc24a2413289605d92553b0b60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb38f9cc24a2413289605d92553b0b60" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 38635 } } peers { permanent_uuid: "98033857a04c4ac794f0feeead2ddc3d" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 41935 } } peers { permanent_uuid: "0e4d7e5ba5744b47a854f00460dac7aa" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 36707 } }
I20260806 11:36:45.760257 29396 raft_consensus.cc:374] T 27f463bfebf745569c3e2a9e9ae37bfd P cb38f9cc24a2413289605d92553b0b60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:45.760324 29396 raft_consensus.cc:729] T 27f463bfebf745569c3e2a9e9ae37bfd P cb38f9cc24a2413289605d92553b0b60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb38f9cc24a2413289605d92553b0b60, State: Initialized, Role: FOLLOWER
I20260806 11:36:45.760310 29395 raft_consensus.cc:348] T 27f463bfebf745569c3e2a9e9ae37bfd P 98033857a04c4ac794f0feeead2ddc3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb38f9cc24a2413289605d92553b0b60" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 38635 } } peers { permanent_uuid: "98033857a04c4ac794f0feeead2ddc3d" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 41935 } } peers { permanent_uuid: "0e4d7e5ba5744b47a854f00460dac7aa" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 36707 } }
I20260806 11:36:45.760601 29395 raft_consensus.cc:374] T 27f463bfebf745569c3e2a9e9ae37bfd P 98033857a04c4ac794f0feeead2ddc3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:45.760663 29395 raft_consensus.cc:729] T 27f463bfebf745569c3e2a9e9ae37bfd P 98033857a04c4ac794f0feeead2ddc3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98033857a04c4ac794f0feeead2ddc3d, State: Initialized, Role: FOLLOWER
I20260806 11:36:45.760881 29396 consensus_queue.cc:260] T 27f463bfebf745569c3e2a9e9ae37bfd P cb38f9cc24a2413289605d92553b0b60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb38f9cc24a2413289605d92553b0b60" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 38635 } } peers { permanent_uuid: "98033857a04c4ac794f0feeead2ddc3d" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 41935 } } peers { permanent_uuid: "0e4d7e5ba5744b47a854f00460dac7aa" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 36707 } }
I20260806 11:36:45.761118 29395 consensus_queue.cc:260] T 27f463bfebf745569c3e2a9e9ae37bfd P 98033857a04c4ac794f0feeead2ddc3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb38f9cc24a2413289605d92553b0b60" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 38635 } } peers { permanent_uuid: "98033857a04c4ac794f0feeead2ddc3d" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 41935 } } peers { permanent_uuid: "0e4d7e5ba5744b47a854f00460dac7aa" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 36707 } }
I20260806 11:36:45.762189 29396 ts_tablet_manager.cc:1428] T 27f463bfebf745569c3e2a9e9ae37bfd P cb38f9cc24a2413289605d92553b0b60: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 11:36:45.762269 29395 ts_tablet_manager.cc:1428] T 27f463bfebf745569c3e2a9e9ae37bfd P 98033857a04c4ac794f0feeead2ddc3d: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260806 11:36:45.762557 29360 heartbeater.cc:503] Master 127.21.14.254:46157 was elected leader, sending a full tablet report...
I20260806 11:36:45.763839 29397 raft_consensus.cc:348] T 27f463bfebf745569c3e2a9e9ae37bfd P 0e4d7e5ba5744b47a854f00460dac7aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb38f9cc24a2413289605d92553b0b60" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 38635 } } peers { permanent_uuid: "98033857a04c4ac794f0feeead2ddc3d" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 41935 } } peers { permanent_uuid: "0e4d7e5ba5744b47a854f00460dac7aa" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 36707 } }
I20260806 11:36:45.764139 29397 raft_consensus.cc:374] T 27f463bfebf745569c3e2a9e9ae37bfd P 0e4d7e5ba5744b47a854f00460dac7aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:45.764192 29397 raft_consensus.cc:729] T 27f463bfebf745569c3e2a9e9ae37bfd P 0e4d7e5ba5744b47a854f00460dac7aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e4d7e5ba5744b47a854f00460dac7aa, State: Initialized, Role: FOLLOWER
I20260806 11:36:45.764721 29397 consensus_queue.cc:260] T 27f463bfebf745569c3e2a9e9ae37bfd P 0e4d7e5ba5744b47a854f00460dac7aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb38f9cc24a2413289605d92553b0b60" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 38635 } } peers { permanent_uuid: "98033857a04c4ac794f0feeead2ddc3d" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 41935 } } peers { permanent_uuid: "0e4d7e5ba5744b47a854f00460dac7aa" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 36707 } }
I20260806 11:36:45.765620 29071 heartbeater.cc:503] Master 127.21.14.254:46157 was elected leader, sending a full tablet report...
I20260806 11:36:45.765882 29397 ts_tablet_manager.cc:1428] T 27f463bfebf745569c3e2a9e9ae37bfd P 0e4d7e5ba5744b47a854f00460dac7aa: Time spent starting tablet: real 0.009s user 0.008s sys 0.000s
I20260806 11:36:45.766398 29219 heartbeater.cc:503] Master 127.21.14.254:46157 was elected leader, sending a full tablet report...
I20260806 11:36:45.807965 29403 raft_consensus.cc:482] T 27f463bfebf745569c3e2a9e9ae37bfd P 98033857a04c4ac794f0feeead2ddc3d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 11:36:45.808243 29403 raft_consensus.cc:504] T 27f463bfebf745569c3e2a9e9ae37bfd P 98033857a04c4ac794f0feeead2ddc3d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb38f9cc24a2413289605d92553b0b60" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 38635 } } peers { permanent_uuid: "98033857a04c4ac794f0feeead2ddc3d" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 41935 } } peers { permanent_uuid: "0e4d7e5ba5744b47a854f00460dac7aa" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 36707 } }
I20260806 11:36:45.809665 29403 leader_election.cc:288] T 27f463bfebf745569c3e2a9e9ae37bfd P 98033857a04c4ac794f0feeead2ddc3d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb38f9cc24a2413289605d92553b0b60 (127.21.14.193:38635), 0e4d7e5ba5744b47a854f00460dac7aa (127.21.14.194:36707)
I20260806 11:36:45.814602 29364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:45.810093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.193:38635 (local address 127.21.14.195:56307)
0806 11:36:45.810546 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:45.810560 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:45.810647 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:45.811047 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:45.811051 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:45.811072 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:36:45.811174 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:45.811184 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.812156 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.812161 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:45.813136 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:45.813149 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.814014 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.814019 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:45.814172 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:45.814216 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 11:36:45.814308 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 11:36:45.814424 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"mutex_wait_us":248}
I20260806 11:36:45.814651 29391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:45.810284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:56307 (local address 127.21.14.193:38635)
0806 11:36:45.810427 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:45.810431 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:45.810635 (+ 204us) server_negotiation.cc:407] Connection header received
0806 11:36:45.810782 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:45.810785 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:45.810837 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:45.810931 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:36:45.811306 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.812008 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.813341 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.813832 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.813948 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:45.813998 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:45.814360 (+ 362us) server_negotiation.cc:299] Negotiation successful
0806 11:36:45.814468 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 11:36:45.814795 29363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:45.810045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.194:36707 (local address 127.21.14.195:60523)
0806 11:36:45.810300 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:45.810318 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:45.810477 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:45.810931 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:45.810936 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:45.810959 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:36:45.811107 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:45.811115 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.812129 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.812136 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:45.813319 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:45.813334 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.814350 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.814354 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:45.814468 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:45.814492 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 11:36:45.814583 (+ 91us) client_negotiation.cc:240] Negotiation successful
0806 11:36:45.814659 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"mutex_wait_us":26}
I20260806 11:36:45.814914 29394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:45.810143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:60523 (local address 127.21.14.194:36707)
0806 11:36:45.810306 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:45.810311 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:45.810501 (+ 190us) server_negotiation.cc:407] Connection header received
0806 11:36:45.810579 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:45.810584 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:45.810671 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:45.810755 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:36:45.811251 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.812014 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.813507 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.814146 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.814319 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:45.814369 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:45.814624 (+ 255us) server_negotiation.cc:299] Negotiation successful
0806 11:36:45.814732 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 11:36:45.815564 29019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27f463bfebf745569c3e2a9e9ae37bfd" candidate_uuid: "98033857a04c4ac794f0feeead2ddc3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb38f9cc24a2413289605d92553b0b60" is_pre_election: true
I20260806 11:36:45.815567 29169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27f463bfebf745569c3e2a9e9ae37bfd" candidate_uuid: "98033857a04c4ac794f0feeead2ddc3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e4d7e5ba5744b47a854f00460dac7aa" is_pre_election: true
I20260806 11:36:45.815927 29019 raft_consensus.cc:2454] T 27f463bfebf745569c3e2a9e9ae37bfd P cb38f9cc24a2413289605d92553b0b60 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98033857a04c4ac794f0feeead2ddc3d in term 0.
I20260806 11:36:45.815946 29169 raft_consensus.cc:2454] T 27f463bfebf745569c3e2a9e9ae37bfd P 0e4d7e5ba5744b47a854f00460dac7aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98033857a04c4ac794f0feeead2ddc3d in term 0.
I20260806 11:36:45.816758 29245 leader_election.cc:302] T 27f463bfebf745569c3e2a9e9ae37bfd P 98033857a04c4ac794f0feeead2ddc3d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0e4d7e5ba5744b47a854f00460dac7aa, 98033857a04c4ac794f0feeead2ddc3d, cb38f9cc24a2413289605d92553b0b60; no voters:
I20260806 11:36:45.817306 29403 raft_consensus.cc:2781] T 27f463bfebf745569c3e2a9e9ae37bfd P 98033857a04c4ac794f0feeead2ddc3d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 11:36:45.817435 29403 raft_consensus.cc:482] T 27f463bfebf745569c3e2a9e9ae37bfd P 98033857a04c4ac794f0feeead2ddc3d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 11:36:45.817610 29403 raft_consensus.cc:3037] T 27f463bfebf745569c3e2a9e9ae37bfd P 98033857a04c4ac794f0feeead2ddc3d [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:36:45.819373 29403 raft_consensus.cc:504] T 27f463bfebf745569c3e2a9e9ae37bfd P 98033857a04c4ac794f0feeead2ddc3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb38f9cc24a2413289605d92553b0b60" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 38635 } } peers { permanent_uuid: "98033857a04c4ac794f0feeead2ddc3d" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 41935 } } peers { permanent_uuid: "0e4d7e5ba5744b47a854f00460dac7aa" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 36707 } }
I20260806 11:36:45.819978 29403 leader_election.cc:288] T 27f463bfebf745569c3e2a9e9ae37bfd P 98033857a04c4ac794f0feeead2ddc3d [CANDIDATE]: Term 1 election: Requested vote from peers cb38f9cc24a2413289605d92553b0b60 (127.21.14.193:38635), 0e4d7e5ba5744b47a854f00460dac7aa (127.21.14.194:36707)
I20260806 11:36:45.820636 29019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27f463bfebf745569c3e2a9e9ae37bfd" candidate_uuid: "98033857a04c4ac794f0feeead2ddc3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb38f9cc24a2413289605d92553b0b60"
I20260806 11:36:45.820770 29169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27f463bfebf745569c3e2a9e9ae37bfd" candidate_uuid: "98033857a04c4ac794f0feeead2ddc3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e4d7e5ba5744b47a854f00460dac7aa"
I20260806 11:36:45.820819 29019 raft_consensus.cc:3037] T 27f463bfebf745569c3e2a9e9ae37bfd P cb38f9cc24a2413289605d92553b0b60 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:36:45.820960 29169 raft_consensus.cc:3037] T 27f463bfebf745569c3e2a9e9ae37bfd P 0e4d7e5ba5744b47a854f00460dac7aa [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:36:45.822172 29169 raft_consensus.cc:2454] T 27f463bfebf745569c3e2a9e9ae37bfd P 0e4d7e5ba5744b47a854f00460dac7aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98033857a04c4ac794f0feeead2ddc3d in term 1.
I20260806 11:36:45.822184 29019 raft_consensus.cc:2454] T 27f463bfebf745569c3e2a9e9ae37bfd P cb38f9cc24a2413289605d92553b0b60 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98033857a04c4ac794f0feeead2ddc3d in term 1.
I20260806 11:36:45.822639 29242 leader_election.cc:302] T 27f463bfebf745569c3e2a9e9ae37bfd P 98033857a04c4ac794f0feeead2ddc3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e4d7e5ba5744b47a854f00460dac7aa, 98033857a04c4ac794f0feeead2ddc3d; no voters:
I20260806 11:36:45.823067 29403 raft_consensus.cc:2781] T 27f463bfebf745569c3e2a9e9ae37bfd P 98033857a04c4ac794f0feeead2ddc3d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:36:45.823369 29403 raft_consensus.cc:686] T 27f463bfebf745569c3e2a9e9ae37bfd P 98033857a04c4ac794f0feeead2ddc3d [term 1 LEADER]: Becoming Leader. State: Replica: 98033857a04c4ac794f0feeead2ddc3d, State: Running, Role: LEADER
I20260806 11:36:45.823905 29403 consensus_queue.cc:237] T 27f463bfebf745569c3e2a9e9ae37bfd P 98033857a04c4ac794f0feeead2ddc3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb38f9cc24a2413289605d92553b0b60" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 38635 } } peers { permanent_uuid: "98033857a04c4ac794f0feeead2ddc3d" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 41935 } } peers { permanent_uuid: "0e4d7e5ba5744b47a854f00460dac7aa" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 36707 } }
I20260806 11:36:45.828897 28038 catalog_manager.cc:5495] T 27f463bfebf745569c3e2a9e9ae37bfd P 98033857a04c4ac794f0feeead2ddc3d reported cstate change: term changed from 0 to 1, leader changed from <none> to 98033857a04c4ac794f0feeead2ddc3d (127.21.14.195). New cstate: current_term: 1 leader_uuid: "98033857a04c4ac794f0feeead2ddc3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb38f9cc24a2413289605d92553b0b60" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 38635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98033857a04c4ac794f0feeead2ddc3d" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 41935 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e4d7e5ba5744b47a854f00460dac7aa" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 36707 } health_report { overall_health: UNKNOWN } } }
I20260806 11:36:45.866065 29389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:45.862247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33788 (local address 127.21.14.195:41935)
0806 11:36:45.862415 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:45.862421 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:45.862440 (+ 19us) server_negotiation.cc:407] Connection header received
0806 11:36:45.862542 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:45.862546 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:45.862604 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:45.862729 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 11:36:45.863022 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.863540 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.864450 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.864679 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.864801 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:45.865153 (+ 352us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 11:36:45.865619 (+ 466us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 11:36:45.865763 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:45.865828 (+ 65us) server_negotiation.cc:299] Negotiation successful
0806 11:36:45.865896 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 11:36:45.872833 29019 raft_consensus.cc:1261] T 27f463bfebf745569c3e2a9e9ae37bfd P cb38f9cc24a2413289605d92553b0b60 [term 1 FOLLOWER]: Refusing update from remote peer 98033857a04c4ac794f0feeead2ddc3d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 11:36:45.872833 29169 raft_consensus.cc:1261] T 27f463bfebf745569c3e2a9e9ae37bfd P 0e4d7e5ba5744b47a854f00460dac7aa [term 1 FOLLOWER]: Refusing update from remote peer 98033857a04c4ac794f0feeead2ddc3d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 11:36:45.873718 29403 consensus_queue.cc:1035] T 27f463bfebf745569c3e2a9e9ae37bfd P 98033857a04c4ac794f0feeead2ddc3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb38f9cc24a2413289605d92553b0b60" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 38635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 11:36:45.873965 29405 consensus_queue.cc:1035] T 27f463bfebf745569c3e2a9e9ae37bfd P 98033857a04c4ac794f0feeead2ddc3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e4d7e5ba5744b47a854f00460dac7aa" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 36707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 11:36:45.886317 29410 mvcc.cc:205] Tried to move back new op lower bound from 7315522379246374912 to 7315522379058442240. Current Snapshot: MvccSnapshot[applied={T|T < 7315522379246374912 or (T in {7315522379246374912})}]
I20260806 11:36:45.896854 28038 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40582:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 11:36:45.897086 28038 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40582:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 11:36:45.902153 28038 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f90d94258dce47129eb3ffe48021a93f: Sending DeleteTablet for 3 replicas of tablet 27f463bfebf745569c3e2a9e9ae37bfd
I20260806 11:36:45.903244 28997 tablet_service.cc:1514] Processing DeleteTablet for tablet 27f463bfebf745569c3e2a9e9ae37bfd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:36:45 UTC) from {username='oryx', principal='oryx/127.21.14.254@KRBTEST.COM'} at 127.0.0.1:44350
I20260806 11:36:45.904116 29145 tablet_service.cc:1514] Processing DeleteTablet for tablet 27f463bfebf745569c3e2a9e9ae37bfd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:36:45 UTC) from {username='oryx', principal='oryx/127.21.14.254@KRBTEST.COM'} at 127.0.0.1:34088
I20260806 11:36:45.904242 29291 tablet_service.cc:1514] Processing DeleteTablet for tablet 27f463bfebf745569c3e2a9e9ae37bfd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:36:45 UTC) from {username='oryx', principal='oryx/127.21.14.254@KRBTEST.COM'} at 127.0.0.1:33782
I20260806 11:36:45.904702 29419 tablet_replica.cc:330] T 27f463bfebf745569c3e2a9e9ae37bfd P 98033857a04c4ac794f0feeead2ddc3d: stopping tablet replica
I20260806 11:36:45.905134 29417 tablet_replica.cc:330] T 27f463bfebf745569c3e2a9e9ae37bfd P cb38f9cc24a2413289605d92553b0b60: stopping tablet replica
I20260806 11:36:45.905354 29419 raft_consensus.cc:2229] T 27f463bfebf745569c3e2a9e9ae37bfd P 98033857a04c4ac794f0feeead2ddc3d [term 1 LEADER]: Raft consensus shutting down.
I20260806 11:36:45.905459 29417 raft_consensus.cc:2229] T 27f463bfebf745569c3e2a9e9ae37bfd P cb38f9cc24a2413289605d92553b0b60 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 11:36:45.905792 29419 raft_consensus.cc:2258] T 27f463bfebf745569c3e2a9e9ae37bfd P 98033857a04c4ac794f0feeead2ddc3d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 11:36:45.905825 29417 raft_consensus.cc:2258] T 27f463bfebf745569c3e2a9e9ae37bfd P cb38f9cc24a2413289605d92553b0b60 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 11:36:45.907552 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 28004
I20260806 11:36:45.909507 29418 tablet_replica.cc:330] T 27f463bfebf745569c3e2a9e9ae37bfd P 0e4d7e5ba5744b47a854f00460dac7aa: stopping tablet replica
I20260806 11:36:45.910045 29418 raft_consensus.cc:2229] T 27f463bfebf745569c3e2a9e9ae37bfd P 0e4d7e5ba5744b47a854f00460dac7aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 11:36:45.910528 29418 raft_consensus.cc:2258] T 27f463bfebf745569c3e2a9e9ae37bfd P 0e4d7e5ba5744b47a854f00460dac7aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 11:36:45.911011 29417 ts_tablet_manager.cc:1905] T 27f463bfebf745569c3e2a9e9ae37bfd P cb38f9cc24a2413289605d92553b0b60: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 11:36:45.912151 29419 ts_tablet_manager.cc:1905] T 27f463bfebf745569c3e2a9e9ae37bfd P 98033857a04c4ac794f0feeead2ddc3d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 11:36:45.912468 29418 ts_tablet_manager.cc:1905] T 27f463bfebf745569c3e2a9e9ae37bfd P 0e4d7e5ba5744b47a854f00460dac7aa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 11:36:45.915300 29419 ts_tablet_manager.cc:1918] T 27f463bfebf745569c3e2a9e9ae37bfd P 98033857a04c4ac794f0feeead2ddc3d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 11:36:45.915421 29419 log.cc:1196] T 27f463bfebf745569c3e2a9e9ae37bfd P 98033857a04c4ac794f0feeead2ddc3d: Deleting WAL directory at /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-2/wal/wals/27f463bfebf745569c3e2a9e9ae37bfd
I20260806 11:36:45.915784 29419 ts_tablet_manager.cc:1939] T 27f463bfebf745569c3e2a9e9ae37bfd P 98033857a04c4ac794f0feeead2ddc3d: Deleting consensus metadata
I20260806 11:36:45.916332 29418 ts_tablet_manager.cc:1918] T 27f463bfebf745569c3e2a9e9ae37bfd P 0e4d7e5ba5744b47a854f00460dac7aa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 11:36:45.916450 29418 log.cc:1196] T 27f463bfebf745569c3e2a9e9ae37bfd P 0e4d7e5ba5744b47a854f00460dac7aa: Deleting WAL directory at /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-1/wal/wals/27f463bfebf745569c3e2a9e9ae37bfd
I20260806 11:36:45.916824 29418 ts_tablet_manager.cc:1939] T 27f463bfebf745569c3e2a9e9ae37bfd P 0e4d7e5ba5744b47a854f00460dac7aa: Deleting consensus metadata
I20260806 11:36:45.919809 29417 ts_tablet_manager.cc:1918] T 27f463bfebf745569c3e2a9e9ae37bfd P cb38f9cc24a2413289605d92553b0b60: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 11:36:45.919977 29417 log.cc:1196] T 27f463bfebf745569c3e2a9e9ae37bfd P cb38f9cc24a2413289605d92553b0b60: Deleting WAL directory at /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-0/wal/wals/27f463bfebf745569c3e2a9e9ae37bfd
I20260806 11:36:45.920358 29417 ts_tablet_manager.cc:1939] T 27f463bfebf745569c3e2a9e9ae37bfd P cb38f9cc24a2413289605d92553b0b60: Deleting consensus metadata
W20260806 11:36:45.922287 28951 connection.cc:535] server connection from 127.0.0.1:44350 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 11:36:45.922305 29098 connection.cc:535] server connection from 127.0.0.1:34088 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 11:36:45.923059 29245 connection.cc:535] server connection from 127.0.0.1:33782 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 11:36:45.936923 29371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:45.906449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34256 (local address 127.21.14.253:43931)
0806 11:36:45.906598 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:45.906602 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:45.907239 (+ 637us) server_negotiation.cc:407] Connection header received
0806 11:36:45.907432 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:45.907435 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:45.907497 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:45.907585 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:45.920275 (+ 12690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.920929 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.921948 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.922340 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.922453 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:45.930537 (+ 8084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:45.930542 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:45.930546 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:45.930585 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:45.932873 (+ 2288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:45.933722 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:45.933726 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:45.933728 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:45.933826 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:45.934437 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:45.934441 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:45.934442 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:45.934693 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:45.934807 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:45.936577 (+ 1770us) server_negotiation.cc:299] Negotiation successful
0806 11:36:45.936733 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 11:36:45.936942 29076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:45.926694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32850 (local address 127.21.14.252:40757)
0806 11:36:45.926809 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:45.926813 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:45.926828 (+ 15us) server_negotiation.cc:407] Connection header received
0806 11:36:45.926991 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:45.926995 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:45.927082 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:45.927135 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:45.928093 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.928926 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.930785 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.931205 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.931317 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:45.933061 (+ 1744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:45.933065 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:45.933068 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:45.933098 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:45.934909 (+ 1811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:45.935501 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:45.935505 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:45.935507 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:45.935599 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:45.935900 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:45.935904 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:45.935906 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:45.936123 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:45.936233 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:45.936651 (+ 418us) server_negotiation.cc:299] Negotiation successful
0806 11:36:45.936782 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 11:36:45.940776 28815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:45.926194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34266 (local address 127.21.14.253:43931)
0806 11:36:45.926339 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:45.926343 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:45.931230 (+ 4887us) server_negotiation.cc:407] Connection header received
0806 11:36:45.931447 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:45.931453 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:45.931512 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:45.931575 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:45.932584 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.933401 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.934644 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.934991 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.935080 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:45.936842 (+ 1762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:45.936845 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:45.936848 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:45.936870 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:45.938645 (+ 1775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:45.939244 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:45.939248 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:45.939250 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:45.939416 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:45.939747 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:45.939751 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:45.939752 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:45.939983 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:45.940113 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:45.940445 (+ 332us) server_negotiation.cc:299] Negotiation successful
0806 11:36:45.940573 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 11:36:45.947165 29374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:45.906014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32848 (local address 127.21.14.252:40757)
0806 11:36:45.906139 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:45.906144 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:45.935932 (+ 29788us) server_negotiation.cc:407] Connection header received
0806 11:36:45.936170 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:45.936175 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:45.936245 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:45.936342 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:45.939319 (+ 2977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.939963 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.941135 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:45.941443 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:45.941563 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:45.943452 (+ 1889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:45.943456 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:45.943458 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:45.943488 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:45.945142 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:45.945652 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:45.945656 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:45.945660 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:45.945752 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:45.946069 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:45.946073 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:45.946075 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:45.946286 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:45.946434 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:45.946700 (+ 266us) server_negotiation.cc:299] Negotiation successful
0806 11:36:45.946935 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 11:36:46.888981 29428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:46.888179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:46157 (local address 127.21.14.193:56973)
0806 11:36:46.888732 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:46.888837 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:46157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":89,"threads_started":1}
W20260806 11:36:46.889889 29071 heartbeater.cc:644] Failed to heartbeat to 127.21.14.254:46157 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.21.14.254:46157: connect: Connection refused (error 111)
I20260806 11:36:46.890630 29429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:46.890107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:46157 (local address 127.21.14.194:45163)
0806 11:36:46.890427 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:46.890508 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:46157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":99,"threads_started":1}
W20260806 11:36:46.890920 29219 heartbeater.cc:644] Failed to heartbeat to 127.21.14.254:46157 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.21.14.254:46157: connect: Connection refused (error 111)
I20260806 11:36:46.891703 29430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:46.891302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:46157 (local address 127.21.14.195:51781)
0806 11:36:46.891551 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:46.891624 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:46157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":66,"threads_started":1}
W20260806 11:36:46.891969 29360 heartbeater.cc:644] Failed to heartbeat to 127.21.14.254:46157 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.21.14.254:46157: connect: Connection refused (error 111)
I20260806 11:36:47.403298 29431 raft_consensus.cc:482] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f90d94258dce47129eb3ffe48021a93f)
I20260806 11:36:47.403575 29431 raft_consensus.cc:504] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f90d94258dce47129eb3ffe48021a93f" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 46157 } } peers { permanent_uuid: "b160eb10619648eeacfcc3d72bae9fd4" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 43931 } attrs { promote: false } } peers { permanent_uuid: "9e5a72b344fe41a1ae5787da75d1b90f" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 40757 } attrs { promote: false } }
I20260806 11:36:47.404089 29432 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f90d94258dce47129eb3ffe48021a93f)
I20260806 11:36:47.404261 29432 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f90d94258dce47129eb3ffe48021a93f" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 46157 } } peers { permanent_uuid: "b160eb10619648eeacfcc3d72bae9fd4" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 43931 } attrs { promote: false } } peers { permanent_uuid: "9e5a72b344fe41a1ae5787da75d1b90f" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 40757 } attrs { promote: false } }
I20260806 11:36:47.404929 29431 leader_election.cc:288] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f90d94258dce47129eb3ffe48021a93f (127.21.14.254:46157), 9e5a72b344fe41a1ae5787da75d1b90f (127.21.14.252:40757)
I20260806 11:36:47.405241 29432 leader_election.cc:288] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f90d94258dce47129eb3ffe48021a93f (127.21.14.254:46157), b160eb10619648eeacfcc3d72bae9fd4 (127.21.14.253:43931)
I20260806 11:36:47.405762 29433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:47.405183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:46157 (local address 127.0.0.1:40990)
0806 11:36:47.405512 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:47.405644 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:46157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":88,"threads_started":1}
W20260806 11:36:47.406210 28669 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.14.254:46157: connect: Connection refused (error 111)
I20260806 11:36:47.406476 29434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:47.405390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:46157 (local address 127.0.0.1:40992)
0806 11:36:47.405792 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:47.406359 (+ 567us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:46157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328,"spinlock_wait_cycles":9984,"thread_start_us":73,"threads_started":1}
W20260806 11:36:47.406781 28865 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.14.254:46157: connect: Connection refused (error 111)
I20260806 11:36:47.407279 29437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:47.406983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:46157 (local address 127.0.0.1:41000)
0806 11:36:47.407141 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:47.407194 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:46157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
W20260806 11:36:47.407449 28669 leader_election.cc:334] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f90d94258dce47129eb3ffe48021a93f (127.21.14.254:46157): Network error: Client connection negotiation failed: client connection to 127.21.14.254:46157: connect: Connection refused (error 111)
I20260806 11:36:47.407976 29434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:47.407742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:46157 (local address 127.0.0.1:41014)
0806 11:36:47.407855 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:47.407909 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:46157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260806 11:36:47.408138 28865 leader_election.cc:334] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f90d94258dce47129eb3ffe48021a93f (127.21.14.254:46157): Network error: Client connection negotiation failed: client connection to 127.21.14.254:46157: connect: Connection refused (error 111)
I20260806 11:36:47.410184 29438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:47.405728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34272 (local address 127.21.14.253:43931)
0806 11:36:47.406003 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:47.406010 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:47.406173 (+ 163us) server_negotiation.cc:407] Connection header received
0806 11:36:47.406231 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:47.406235 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:47.406281 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:47.406361 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:36:47.406782 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:47.407623 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:47.409111 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:47.409551 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:47.409685 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:47.409754 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:47.409955 (+ 201us) server_negotiation.cc:299] Negotiation successful
0806 11:36:47.410039 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":66,"threads_started":1}
I20260806 11:36:47.410185 29435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:47.405412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.253:43931 (local address 127.0.0.1:34272)
0806 11:36:47.406001 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:47.406014 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:47.406149 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:47.406495 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:47.406501 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:47.406525 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:36:47.406636 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:47.406646 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:47.407764 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:47.407770 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:47.408964 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:47.408976 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:47.409701 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:47.409705 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:47.409806 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:47.409842 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 11:36:47.409936 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 11:36:47.410019 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"mutex_wait_us":1,"thread_start_us":78,"threads_started":1}
I20260806 11:36:47.410362 29433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:47.405537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.252:40757 (local address 127.0.0.1:32866)
0806 11:36:47.405928 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:47.405942 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:47.406063 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:47.406487 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:47.406494 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:47.406518 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:36:47.406626 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:47.406635 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:47.407930 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:47.407935 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:47.409099 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:47.409110 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:47.409796 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:47.409800 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:47.409898 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:47.409926 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 11:36:47.410030 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 11:36:47.410126 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":76,"threads_started":1}
I20260806 11:36:47.410562 29439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:47.405922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32866 (local address 127.21.14.252:40757)
0806 11:36:47.406162 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:47.406166 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:47.406181 (+ 15us) server_negotiation.cc:407] Connection header received
0806 11:36:47.406228 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:47.406231 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:47.406274 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:47.406357 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:36:47.406763 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:47.407824 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:47.409246 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:47.409661 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:47.409790 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:47.409893 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:47.410051 (+ 158us) server_negotiation.cc:299] Negotiation successful
0806 11:36:47.410140 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":186,"thread_start_us":70,"threads_started":1}
I20260806 11:36:47.410821 28692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9e5a72b344fe41a1ae5787da75d1b90f" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "b160eb10619648eeacfcc3d72bae9fd4" is_pre_election: true
I20260806 11:36:47.411125 28692 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e5a72b344fe41a1ae5787da75d1b90f in term 1.
I20260806 11:36:47.411098 28890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b160eb10619648eeacfcc3d72bae9fd4" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "9e5a72b344fe41a1ae5787da75d1b90f" is_pre_election: true
I20260806 11:36:47.411283 28890 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b160eb10619648eeacfcc3d72bae9fd4 in term 1.
I20260806 11:36:47.411582 28863 leader_election.cc:302] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e5a72b344fe41a1ae5787da75d1b90f, b160eb10619648eeacfcc3d72bae9fd4; no voters: f90d94258dce47129eb3ffe48021a93f
I20260806 11:36:47.411841 29432 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 11:36:47.411919 29432 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f [term 1 FOLLOWER]: Starting leader election (detected failure of leader f90d94258dce47129eb3ffe48021a93f)
I20260806 11:36:47.411952 29432 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f [term 1 FOLLOWER]: Advancing to term 2
I20260806 11:36:47.412140 28669 leader_election.cc:302] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e5a72b344fe41a1ae5787da75d1b90f, b160eb10619648eeacfcc3d72bae9fd4; no voters: f90d94258dce47129eb3ffe48021a93f
I20260806 11:36:47.412354 29431 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 11:36:47.412411 29431 raft_consensus.cc:482] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f90d94258dce47129eb3ffe48021a93f)
I20260806 11:36:47.412451 29431 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4 [term 1 FOLLOWER]: Advancing to term 2
I20260806 11:36:47.413470 29432 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f90d94258dce47129eb3ffe48021a93f" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 46157 } } peers { permanent_uuid: "b160eb10619648eeacfcc3d72bae9fd4" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 43931 } attrs { promote: false } } peers { permanent_uuid: "9e5a72b344fe41a1ae5787da75d1b90f" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 40757 } attrs { promote: false } }
I20260806 11:36:47.413767 29431 raft_consensus.cc:504] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f90d94258dce47129eb3ffe48021a93f" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 46157 } } peers { permanent_uuid: "b160eb10619648eeacfcc3d72bae9fd4" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 43931 } attrs { promote: false } } peers { permanent_uuid: "9e5a72b344fe41a1ae5787da75d1b90f" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 40757 } attrs { promote: false } }
I20260806 11:36:47.413993 29432 leader_election.cc:288] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f [CANDIDATE]: Term 2 election: Requested vote from peers f90d94258dce47129eb3ffe48021a93f (127.21.14.254:46157), b160eb10619648eeacfcc3d72bae9fd4 (127.21.14.253:43931)
I20260806 11:36:47.414203 29431 leader_election.cc:288] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4 [CANDIDATE]: Term 2 election: Requested vote from peers f90d94258dce47129eb3ffe48021a93f (127.21.14.254:46157), 9e5a72b344fe41a1ae5787da75d1b90f (127.21.14.252:40757)
I20260806 11:36:47.414443 28692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9e5a72b344fe41a1ae5787da75d1b90f" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "b160eb10619648eeacfcc3d72bae9fd4"
I20260806 11:36:47.414630 28692 raft_consensus.cc:2379] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e5a72b344fe41a1ae5787da75d1b90f in current term 2: Already voted for candidate b160eb10619648eeacfcc3d72bae9fd4 in this term.
I20260806 11:36:47.414701 28890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b160eb10619648eeacfcc3d72bae9fd4" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "9e5a72b344fe41a1ae5787da75d1b90f"
I20260806 11:36:47.414796 29433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:47.414299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:46157 (local address 127.0.0.1:41040)
0806 11:36:47.414639 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:47.414708 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:46157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232}
I20260806 11:36:47.414904 28890 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate b160eb10619648eeacfcc3d72bae9fd4 in current term 2: Already voted for candidate 9e5a72b344fe41a1ae5787da75d1b90f in this term.
I20260806 11:36:47.415472 29435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:47.414280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:46157 (local address 127.0.0.1:41026)
0806 11:36:47.415286 (+ 1006us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:47.415356 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:46157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":925}
I20260806 11:36:47.415712 29437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:47.415429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:46157 (local address 127.0.0.1:41056)
0806 11:36:47.415592 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:47.415656 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:46157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260806 11:36:47.415889 28669 leader_election.cc:334] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer f90d94258dce47129eb3ffe48021a93f (127.21.14.254:46157): Network error: Client connection negotiation failed: client connection to 127.21.14.254:46157: connect: Connection refused (error 111)
I20260806 11:36:47.415956 28669 leader_election.cc:302] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b160eb10619648eeacfcc3d72bae9fd4; no voters: 9e5a72b344fe41a1ae5787da75d1b90f, f90d94258dce47129eb3ffe48021a93f
I20260806 11:36:47.416220 29431 raft_consensus.cc:2726] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260806 11:36:47.416234 29435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:47.416000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:46157 (local address 127.0.0.1:41058)
0806 11:36:47.416124 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:47.416180 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:46157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260806 11:36:47.416363 28865 leader_election.cc:334] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer f90d94258dce47129eb3ffe48021a93f (127.21.14.254:46157): Network error: Client connection negotiation failed: client connection to 127.21.14.254:46157: connect: Connection refused (error 111)
I20260806 11:36:47.416435 28865 leader_election.cc:302] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e5a72b344fe41a1ae5787da75d1b90f; no voters: b160eb10619648eeacfcc3d72bae9fd4, f90d94258dce47129eb3ffe48021a93f
I20260806 11:36:47.416574 29432 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260806 11:36:47.891501 29442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:47.890766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:46157 (local address 127.21.14.193:56331)
0806 11:36:47.891295 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:47.891378 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:46157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":174,"threads_started":1}
I20260806 11:36:47.892123 29443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:47.891617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:46157 (local address 127.21.14.194:38189)
0806 11:36:47.891947 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:47.892012 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:46157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":87,"threads_started":1}
I20260806 11:36:47.892951 29444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:47.892584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:46157 (local address 127.21.14.195:50427)
0806 11:36:47.892818 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:47.892874 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:46157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":80,"threads_started":1}
I20260806 11:36:48.893002 29445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:48.892253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:46157 (local address 127.21.14.193:49301)
0806 11:36:48.892779 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:48.892872 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:46157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":109,"threads_started":1}
I20260806 11:36:48.893761 29446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:48.892867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:46157 (local address 127.21.14.194:56103)
0806 11:36:48.893581 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:48.893658 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:46157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":94,"threads_started":1}
I20260806 11:36:48.894344 29447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:48.893720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:46157 (local address 127.21.14.195:33895)
0806 11:36:48.894199 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:48.894264 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:46157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":70,"threads_started":1}
I20260806 11:36:48.981410 29448 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 11:36:48.981606 29448 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f90d94258dce47129eb3ffe48021a93f" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 46157 } } peers { permanent_uuid: "b160eb10619648eeacfcc3d72bae9fd4" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 43931 } attrs { promote: false } } peers { permanent_uuid: "9e5a72b344fe41a1ae5787da75d1b90f" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 40757 } attrs { promote: false } }
I20260806 11:36:48.982123 29448 leader_election.cc:288] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f90d94258dce47129eb3ffe48021a93f (127.21.14.254:46157), b160eb10619648eeacfcc3d72bae9fd4 (127.21.14.253:43931)
I20260806 11:36:48.982688 28692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9e5a72b344fe41a1ae5787da75d1b90f" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "b160eb10619648eeacfcc3d72bae9fd4" is_pre_election: true
I20260806 11:36:48.982851 29449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:48.982378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:46157 (local address 127.0.0.1:54876)
0806 11:36:48.982657 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:48.982719 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:46157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":90,"threads_started":1}
I20260806 11:36:48.982905 28692 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e5a72b344fe41a1ae5787da75d1b90f in term 2.
I20260806 11:36:48.983347 28863 leader_election.cc:302] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e5a72b344fe41a1ae5787da75d1b90f, b160eb10619648eeacfcc3d72bae9fd4; no voters:
I20260806 11:36:48.983626 29448 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260806 11:36:48.983704 29448 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 11:36:48.983759 29448 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f [term 2 FOLLOWER]: Advancing to term 3
I20260806 11:36:48.983851 29449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:48.983582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:46157 (local address 127.0.0.1:54890)
0806 11:36:48.983686 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:48.983751 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:46157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260806 11:36:48.983970 28865 leader_election.cc:334] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer f90d94258dce47129eb3ffe48021a93f (127.21.14.254:46157): Network error: Client connection negotiation failed: client connection to 127.21.14.254:46157: connect: Connection refused (error 111)
I20260806 11:36:48.984944 29448 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f90d94258dce47129eb3ffe48021a93f" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 46157 } } peers { permanent_uuid: "b160eb10619648eeacfcc3d72bae9fd4" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 43931 } attrs { promote: false } } peers { permanent_uuid: "9e5a72b344fe41a1ae5787da75d1b90f" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 40757 } attrs { promote: false } }
I20260806 11:36:48.985352 29448 leader_election.cc:288] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f [CANDIDATE]: Term 3 election: Requested vote from peers f90d94258dce47129eb3ffe48021a93f (127.21.14.254:46157), b160eb10619648eeacfcc3d72bae9fd4 (127.21.14.253:43931)
I20260806 11:36:48.985870 29449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:48.985591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:46157 (local address 127.0.0.1:54896)
0806 11:36:48.985729 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:48.985789 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:46157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260806 11:36:48.985852 28692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9e5a72b344fe41a1ae5787da75d1b90f" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "b160eb10619648eeacfcc3d72bae9fd4"
I20260806 11:36:48.986030 28692 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4 [term 2 FOLLOWER]: Advancing to term 3
I20260806 11:36:48.986572 29449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:48.986334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:46157 (local address 127.0.0.1:54900)
0806 11:36:48.986451 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:48.986503 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:46157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260806 11:36:48.986707 28865 leader_election.cc:334] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer f90d94258dce47129eb3ffe48021a93f (127.21.14.254:46157): Network error: Client connection negotiation failed: client connection to 127.21.14.254:46157: connect: Connection refused (error 111)
I20260806 11:36:48.987322 28692 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e5a72b344fe41a1ae5787da75d1b90f in term 3.
I20260806 11:36:48.987681 28863 leader_election.cc:302] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e5a72b344fe41a1ae5787da75d1b90f, b160eb10619648eeacfcc3d72bae9fd4; no voters: f90d94258dce47129eb3ffe48021a93f
I20260806 11:36:48.987926 29448 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f [term 3 FOLLOWER]: Leader election won for term 3
I20260806 11:36:48.988185 29448 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f [term 3 LEADER]: Becoming Leader. State: Replica: 9e5a72b344fe41a1ae5787da75d1b90f, State: Running, Role: LEADER
I20260806 11:36:48.988480 29448 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f90d94258dce47129eb3ffe48021a93f" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 46157 } } peers { permanent_uuid: "b160eb10619648eeacfcc3d72bae9fd4" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 43931 } attrs { promote: false } } peers { permanent_uuid: "9e5a72b344fe41a1ae5787da75d1b90f" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 40757 } attrs { promote: false } }
I20260806 11:36:48.990000 29451 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9e5a72b344fe41a1ae5787da75d1b90f. Latest consensus state: current_term: 3 leader_uuid: "9e5a72b344fe41a1ae5787da75d1b90f" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f90d94258dce47129eb3ffe48021a93f" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 46157 } } peers { permanent_uuid: "b160eb10619648eeacfcc3d72bae9fd4" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 43931 } attrs { promote: false } } peers { permanent_uuid: "9e5a72b344fe41a1ae5787da75d1b90f" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 40757 } attrs { promote: false } } }
I20260806 11:36:48.990175 29451 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f [sys.catalog]: This master's current role is: LEADER
I20260806 11:36:48.990525 29454 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 11:36:48.993108 29454 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 11:36:48.993571 29454 catalog_manager.cc:1259] Loaded cluster ID: f64fef229c0a4d3588ddb9a279aad6dd
I20260806 11:36:48.993626 29454 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 11:36:48.994261 29454 catalog_manager.cc:1504] Loading token signing keys...
I20260806 11:36:48.994692 29454 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f: Loaded TSK: 0
I20260806 11:36:48.995525 29454 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 11:36:49.034969 28878 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32850:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 11:36:49.038993 28878 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 11:36:49.042667 29449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:49.042363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:46157 (local address 127.0.0.1:54918)
0806 11:36:49.042512 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:49.042581 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:46157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260806 11:36:49.043015 28692 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4 [term 3 FOLLOWER]: Refusing update from remote peer 9e5a72b344fe41a1ae5787da75d1b90f: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260806 11:36:49.043726 29449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:49.043382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:46157 (local address 127.0.0.1:54922)
0806 11:36:49.043589 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:49.043656 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:46157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260806 11:36:49.043622 29451 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f [LEADER]: Connected to new peer: Peer: permanent_uuid: "b160eb10619648eeacfcc3d72bae9fd4" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 43931 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260806 11:36:49.043900 28865 consensus_peers.cc:487] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f -> Peer f90d94258dce47129eb3ffe48021a93f (127.21.14.254:46157): Couldn't send request to peer f90d94258dce47129eb3ffe48021a93f. Status: Network error: Client connection negotiation failed: client connection to 127.21.14.254:46157: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 11:36:49.045634 29456 sys_catalog.cc:455] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9e5a72b344fe41a1ae5787da75d1b90f. Latest consensus state: current_term: 3 leader_uuid: "9e5a72b344fe41a1ae5787da75d1b90f" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f90d94258dce47129eb3ffe48021a93f" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 46157 } } peers { permanent_uuid: "b160eb10619648eeacfcc3d72bae9fd4" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 43931 } attrs { promote: false } } peers { permanent_uuid: "9e5a72b344fe41a1ae5787da75d1b90f" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 40757 } attrs { promote: false } } }
I20260806 11:36:49.045775 29456 sys_catalog.cc:458] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 11:36:49.047551 29451 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "9e5a72b344fe41a1ae5787da75d1b90f" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f90d94258dce47129eb3ffe48021a93f" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 46157 } } peers { permanent_uuid: "b160eb10619648eeacfcc3d72bae9fd4" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 43931 } attrs { promote: false } } peers { permanent_uuid: "9e5a72b344fe41a1ae5787da75d1b90f" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 40757 } attrs { promote: false } } }
I20260806 11:36:49.047737 29451 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f [sys.catalog]: This master's current role is: LEADER
I20260806 11:36:49.048327 29456 sys_catalog.cc:455] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "9e5a72b344fe41a1ae5787da75d1b90f" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f90d94258dce47129eb3ffe48021a93f" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 46157 } } peers { permanent_uuid: "b160eb10619648eeacfcc3d72bae9fd4" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 43931 } attrs { promote: false } } peers { permanent_uuid: "9e5a72b344fe41a1ae5787da75d1b90f" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 40757 } attrs { promote: false } } }
I20260806 11:36:49.048502 29456 sys_catalog.cc:458] T 00000000000000000000000000000000 P b160eb10619648eeacfcc3d72bae9fd4 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 11:36:49.060135 29449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:49.054709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.193:38635 (local address 127.0.0.1:57136)
0806 11:36:49.054932 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:49.054949 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:49.055060 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:49.056328 (+ 1268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:49.056331 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:49.056347 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:36:49.056467 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:49.056473 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:49.057309 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:49.057315 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:49.058649 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:49.058660 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:49.059671 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:49.059675 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:49.059765 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:49.059788 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 11:36:49.059868 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 11:36:49.059950 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"mutex_wait_us":57}
I20260806 11:36:49.060137 29462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:49.054807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57136 (local address 127.21.14.193:38635)
0806 11:36:49.056090 (+ 1283us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:49.056094 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:49.056111 (+ 17us) server_negotiation.cc:407] Connection header received
0806 11:36:49.056170 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:49.056172 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:49.056214 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:49.056315 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:36:49.056603 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:49.057186 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:49.058802 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:49.059508 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:49.059668 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:49.059737 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:49.059906 (+ 169us) server_negotiation.cc:299] Negotiation successful
0806 11:36:49.059997 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":665,"thread_start_us":133,"threads_started":1}
I20260806 11:36:49.060286 29467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:49.055603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38688 (local address 127.21.14.194:36707)
0806 11:36:49.055861 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:49.055866 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:49.055881 (+ 15us) server_negotiation.cc:407] Connection header received
0806 11:36:49.055948 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:49.055952 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:49.056002 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:49.056093 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:36:49.056507 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:49.057364 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:49.058676 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:49.059403 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:49.059525 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:49.059597 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:49.060026 (+ 429us) server_negotiation.cc:299] Negotiation successful
0806 11:36:49.060119 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":108,"threads_started":1}
I20260806 11:36:49.060586 29464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:49.054925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.195:41935 (local address 127.0.0.1:36846)
0806 11:36:49.055484 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:49.055499 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:49.055606 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:49.056183 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:49.056187 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:49.056207 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:36:49.056608 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:49.056613 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:49.057638 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:49.057642 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:49.059226 (+ 1584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:49.059236 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:49.060163 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:49.060170 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:49.060264 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:49.060288 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 11:36:49.060345 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 11:36:49.060433 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"mutex_wait_us":638,"thread_start_us":129,"threads_started":1}
I20260806 11:36:49.060587 29463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:49.055229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.194:36707 (local address 127.0.0.1:38688)
0806 11:36:49.055657 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:49.055670 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:49.055761 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:49.056224 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:49.056227 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:49.056239 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:36:49.056370 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:49.056378 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:49.057500 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:49.057503 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:49.058566 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:49.058576 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:49.059553 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:49.059559 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:49.059667 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:49.059696 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 11:36:49.059772 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 11:36:49.059894 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":372,"mutex_wait_us":56,"thread_start_us":105,"threads_started":1}
I20260806 11:36:49.061244 29465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:49.055321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36846 (local address 127.21.14.195:41935)
0806 11:36:49.055919 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:49.055924 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:49.055937 (+ 13us) server_negotiation.cc:407] Connection header received
0806 11:36:49.055986 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:49.055989 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:49.056044 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:49.056373 (+ 329us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:36:49.056726 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:49.057508 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:49.059371 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:49.060025 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:49.060899 (+ 874us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:49.060988 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:49.061025 (+ 37us) server_negotiation.cc:299] Negotiation successful
0806 11:36:49.061107 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":519,"thread_start_us":79,"threads_started":1}
I20260806 11:36:49.061204 29145 tablet_service.cc:1467] Processing CreateTablet for tablet 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 (DEFAULT_TABLE table=test-table [id=da38e5ef3d03490fb85f50f6a5d7b0f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:36:49.061203 28997 tablet_service.cc:1467] Processing CreateTablet for tablet 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 (DEFAULT_TABLE table=test-table [id=da38e5ef3d03490fb85f50f6a5d7b0f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:36:49.061530 29145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a53f1d2dcd44e0dbf11d1f68a7ff8d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:49.061535 28997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a53f1d2dcd44e0dbf11d1f68a7ff8d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:49.064198 29469 tablet_bootstrap.cc:492] T 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 P cb38f9cc24a2413289605d92553b0b60: Bootstrap starting.
I20260806 11:36:49.064786 29291 tablet_service.cc:1467] Processing CreateTablet for tablet 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 (DEFAULT_TABLE table=test-table [id=da38e5ef3d03490fb85f50f6a5d7b0f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:36:49.065107 29291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a53f1d2dcd44e0dbf11d1f68a7ff8d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:49.065358 29469 tablet_bootstrap.cc:654] T 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 P cb38f9cc24a2413289605d92553b0b60: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:49.066172 29468 tablet_bootstrap.cc:492] T 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 P 0e4d7e5ba5744b47a854f00460dac7aa: Bootstrap starting.
I20260806 11:36:49.066710 29469 tablet_bootstrap.cc:492] T 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 P cb38f9cc24a2413289605d92553b0b60: No bootstrap required, opened a new log
I20260806 11:36:49.066816 29469 ts_tablet_manager.cc:1397] T 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 P cb38f9cc24a2413289605d92553b0b60: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 11:36:49.067375 29469 raft_consensus.cc:348] T 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 P cb38f9cc24a2413289605d92553b0b60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb38f9cc24a2413289605d92553b0b60" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 38635 } } peers { permanent_uuid: "98033857a04c4ac794f0feeead2ddc3d" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 41935 } } peers { permanent_uuid: "0e4d7e5ba5744b47a854f00460dac7aa" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 36707 } }
I20260806 11:36:49.067480 29469 raft_consensus.cc:374] T 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 P cb38f9cc24a2413289605d92553b0b60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:49.067503 29469 raft_consensus.cc:729] T 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 P cb38f9cc24a2413289605d92553b0b60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb38f9cc24a2413289605d92553b0b60, State: Initialized, Role: FOLLOWER
I20260806 11:36:49.067647 29469 consensus_queue.cc:260] T 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 P cb38f9cc24a2413289605d92553b0b60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb38f9cc24a2413289605d92553b0b60" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 38635 } } peers { permanent_uuid: "98033857a04c4ac794f0feeead2ddc3d" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 41935 } } peers { permanent_uuid: "0e4d7e5ba5744b47a854f00460dac7aa" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 36707 } }
I20260806 11:36:49.068095 29469 ts_tablet_manager.cc:1428] T 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 P cb38f9cc24a2413289605d92553b0b60: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 11:36:49.068945 29445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:49.068634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:46157 (local address 127.21.14.193:37327)
0806 11:36:49.068792 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:49.068864 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:46157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260806 11:36:49.069371 29468 tablet_bootstrap.cc:654] T 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 P 0e4d7e5ba5744b47a854f00460dac7aa: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:49.071437 29468 tablet_bootstrap.cc:492] T 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 P 0e4d7e5ba5744b47a854f00460dac7aa: No bootstrap required, opened a new log
I20260806 11:36:49.071609 29070 heartbeater.cc:503] Master 127.21.14.252:40757 was elected leader, sending a full tablet report...
I20260806 11:36:49.071636 29468 ts_tablet_manager.cc:1397] T 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 P 0e4d7e5ba5744b47a854f00460dac7aa: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.001s
I20260806 11:36:49.072244 29468 raft_consensus.cc:348] T 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 P 0e4d7e5ba5744b47a854f00460dac7aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb38f9cc24a2413289605d92553b0b60" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 38635 } } peers { permanent_uuid: "98033857a04c4ac794f0feeead2ddc3d" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 41935 } } peers { permanent_uuid: "0e4d7e5ba5744b47a854f00460dac7aa" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 36707 } }
I20260806 11:36:49.072413 29468 raft_consensus.cc:374] T 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 P 0e4d7e5ba5744b47a854f00460dac7aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:49.072479 29468 raft_consensus.cc:729] T 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 P 0e4d7e5ba5744b47a854f00460dac7aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e4d7e5ba5744b47a854f00460dac7aa, State: Initialized, Role: FOLLOWER
I20260806 11:36:49.072671 29468 consensus_queue.cc:260] T 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 P 0e4d7e5ba5744b47a854f00460dac7aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb38f9cc24a2413289605d92553b0b60" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 38635 } } peers { permanent_uuid: "98033857a04c4ac794f0feeead2ddc3d" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 41935 } } peers { permanent_uuid: "0e4d7e5ba5744b47a854f00460dac7aa" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 36707 } }
I20260806 11:36:49.073151 29468 ts_tablet_manager.cc:1428] T 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 P 0e4d7e5ba5744b47a854f00460dac7aa: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 11:36:49.073216 29445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:49.072674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:46157 (local address 127.21.14.193:36641)
0806 11:36:49.073016 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:49.073119 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:46157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":253}
I20260806 11:36:49.074133 29472 tablet_bootstrap.cc:492] T 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 P 98033857a04c4ac794f0feeead2ddc3d: Bootstrap starting.
I20260806 11:36:49.075608 29472 tablet_bootstrap.cc:654] T 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 P 98033857a04c4ac794f0feeead2ddc3d: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:49.075771 29218 heartbeater.cc:503] Master 127.21.14.252:40757 was elected leader, sending a full tablet report...
I20260806 11:36:49.075947 29446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:49.075334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:46157 (local address 127.21.14.194:34879)
0806 11:36:49.075805 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:49.075869 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:46157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":381}
I20260806 11:36:49.077075 29446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:49.076784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:46157 (local address 127.21.14.194:52341)
0806 11:36:49.076937 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:49.077000 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:46157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260806 11:36:49.077216 29472 tablet_bootstrap.cc:492] T 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 P 98033857a04c4ac794f0feeead2ddc3d: No bootstrap required, opened a new log
I20260806 11:36:49.077307 29472 ts_tablet_manager.cc:1397] T 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 P 98033857a04c4ac794f0feeead2ddc3d: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20260806 11:36:49.077761 29472 raft_consensus.cc:348] T 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 P 98033857a04c4ac794f0feeead2ddc3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb38f9cc24a2413289605d92553b0b60" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 38635 } } peers { permanent_uuid: "98033857a04c4ac794f0feeead2ddc3d" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 41935 } } peers { permanent_uuid: "0e4d7e5ba5744b47a854f00460dac7aa" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 36707 } }
I20260806 11:36:49.077893 29472 raft_consensus.cc:374] T 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 P 98033857a04c4ac794f0feeead2ddc3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:49.077922 29472 raft_consensus.cc:729] T 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 P 98033857a04c4ac794f0feeead2ddc3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98033857a04c4ac794f0feeead2ddc3d, State: Initialized, Role: FOLLOWER
I20260806 11:36:49.078058 29472 consensus_queue.cc:260] T 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 P 98033857a04c4ac794f0feeead2ddc3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb38f9cc24a2413289605d92553b0b60" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 38635 } } peers { permanent_uuid: "98033857a04c4ac794f0feeead2ddc3d" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 41935 } } peers { permanent_uuid: "0e4d7e5ba5744b47a854f00460dac7aa" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 36707 } }
I20260806 11:36:49.078459 29472 ts_tablet_manager.cc:1428] T 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 P 98033857a04c4ac794f0feeead2ddc3d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 11:36:49.079296 29447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:49.078997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:46157 (local address 127.21.14.195:51309)
0806 11:36:49.079144 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:49.079214 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:46157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260806 11:36:49.080693 29359 heartbeater.cc:503] Master 127.21.14.252:40757 was elected leader, sending a full tablet report...
I20260806 11:36:49.081107 29447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:49.080783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:46157 (local address 127.21.14.195:34355)
0806 11:36:49.080924 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:49.081022 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:46157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260806 11:36:49.150585 29474 raft_consensus.cc:482] T 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 P 0e4d7e5ba5744b47a854f00460dac7aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 11:36:49.150774 29474 raft_consensus.cc:504] T 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 P 0e4d7e5ba5744b47a854f00460dac7aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb38f9cc24a2413289605d92553b0b60" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 38635 } } peers { permanent_uuid: "98033857a04c4ac794f0feeead2ddc3d" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 41935 } } peers { permanent_uuid: "0e4d7e5ba5744b47a854f00460dac7aa" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 36707 } }
I20260806 11:36:49.152069 29474 leader_election.cc:288] T 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 P 0e4d7e5ba5744b47a854f00460dac7aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb38f9cc24a2413289605d92553b0b60 (127.21.14.193:38635), 98033857a04c4ac794f0feeead2ddc3d (127.21.14.195:41935)
I20260806 11:36:49.156817 29462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:49.152523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:54719 (local address 127.21.14.193:38635)
0806 11:36:49.152695 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:49.152705 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:49.153071 (+ 366us) server_negotiation.cc:407] Connection header received
0806 11:36:49.153160 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:49.153163 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:49.153217 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:49.153322 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:36:49.153751 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:49.154432 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:49.155607 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:49.156041 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:49.156165 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:49.156216 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:49.156571 (+ 355us) server_negotiation.cc:299] Negotiation successful
0806 11:36:49.156670 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260806 11:36:49.156817 29446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:49.152369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.195:41935 (local address 127.21.14.194:36779)
0806 11:36:49.152510 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:49.152527 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:49.152663 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:49.153071 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:49.153074 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:49.153101 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:36:49.153272 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:49.153285 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:49.154230 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:49.154234 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:49.155346 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:49.155360 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:49.156355 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:49.156358 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:49.156455 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:49.156475 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 11:36:49.156585 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 11:36:49.156667 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260806 11:36:49.156826 29465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:49.152460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:36779 (local address 127.21.14.195:41935)
0806 11:36:49.152607 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:49.152614 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:49.152637 (+ 23us) server_negotiation.cc:407] Connection header received
0806 11:36:49.152809 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:49.152814 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:49.152876 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:49.152981 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:36:49.153434 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:49.154078 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:49.155509 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:49.156068 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:49.156352 (+ 284us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:49.156455 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:49.156615 (+ 160us) server_negotiation.cc:299] Negotiation successful
0806 11:36:49.156710 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 11:36:49.156833 29477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:49.152379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.193:38635 (local address 127.21.14.194:54719)
0806 11:36:49.152942 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:49.152960 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:49.153066 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:49.153468 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:49.153471 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:49.153492 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:36:49.153584 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:49.153590 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:49.154568 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:49.154571 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:49.155416 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:49.155448 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:49.156295 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:49.156299 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:49.156408 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:49.156433 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 11:36:49.156524 (+ 91us) client_negotiation.cc:240] Negotiation successful
0806 11:36:49.156645 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"mutex_wait_us":142,"thread_start_us":141,"threads_started":1}
I20260806 11:36:49.157511 29019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a53f1d2dcd44e0dbf11d1f68a7ff8d9" candidate_uuid: "0e4d7e5ba5744b47a854f00460dac7aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb38f9cc24a2413289605d92553b0b60" is_pre_election: true
I20260806 11:36:49.157701 29019 raft_consensus.cc:2454] T 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 P cb38f9cc24a2413289605d92553b0b60 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e4d7e5ba5744b47a854f00460dac7aa in term 0.
I20260806 11:36:49.157876 29303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a53f1d2dcd44e0dbf11d1f68a7ff8d9" candidate_uuid: "0e4d7e5ba5744b47a854f00460dac7aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98033857a04c4ac794f0feeead2ddc3d" is_pre_election: true
I20260806 11:36:49.158178 29303 raft_consensus.cc:2454] T 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 P 98033857a04c4ac794f0feeead2ddc3d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e4d7e5ba5744b47a854f00460dac7aa in term 0.
I20260806 11:36:49.158205 29100 leader_election.cc:302] T 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 P 0e4d7e5ba5744b47a854f00460dac7aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e4d7e5ba5744b47a854f00460dac7aa, cb38f9cc24a2413289605d92553b0b60; no voters:
I20260806 11:36:49.158597 29474 raft_consensus.cc:2781] T 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 P 0e4d7e5ba5744b47a854f00460dac7aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 11:36:49.158687 29474 raft_consensus.cc:482] T 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 P 0e4d7e5ba5744b47a854f00460dac7aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 11:36:49.158749 29474 raft_consensus.cc:3037] T 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 P 0e4d7e5ba5744b47a854f00460dac7aa [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:36:49.159945 29474 raft_consensus.cc:504] T 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 P 0e4d7e5ba5744b47a854f00460dac7aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb38f9cc24a2413289605d92553b0b60" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 38635 } } peers { permanent_uuid: "98033857a04c4ac794f0feeead2ddc3d" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 41935 } } peers { permanent_uuid: "0e4d7e5ba5744b47a854f00460dac7aa" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 36707 } }
I20260806 11:36:49.160369 29474 leader_election.cc:288] T 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 P 0e4d7e5ba5744b47a854f00460dac7aa [CANDIDATE]: Term 1 election: Requested vote from peers cb38f9cc24a2413289605d92553b0b60 (127.21.14.193:38635), 98033857a04c4ac794f0feeead2ddc3d (127.21.14.195:41935)
I20260806 11:36:49.161018 29019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a53f1d2dcd44e0dbf11d1f68a7ff8d9" candidate_uuid: "0e4d7e5ba5744b47a854f00460dac7aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb38f9cc24a2413289605d92553b0b60"
I20260806 11:36:49.161096 29303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a53f1d2dcd44e0dbf11d1f68a7ff8d9" candidate_uuid: "0e4d7e5ba5744b47a854f00460dac7aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98033857a04c4ac794f0feeead2ddc3d"
I20260806 11:36:49.161183 29019 raft_consensus.cc:3037] T 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 P cb38f9cc24a2413289605d92553b0b60 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:36:49.161255 29303 raft_consensus.cc:3037] T 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 P 98033857a04c4ac794f0feeead2ddc3d [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:36:49.162283 29303 raft_consensus.cc:2454] T 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 P 98033857a04c4ac794f0feeead2ddc3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e4d7e5ba5744b47a854f00460dac7aa in term 1.
I20260806 11:36:49.162282 29019 raft_consensus.cc:2454] T 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 P cb38f9cc24a2413289605d92553b0b60 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e4d7e5ba5744b47a854f00460dac7aa in term 1.
I20260806 11:36:49.162791 29100 leader_election.cc:302] T 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 P 0e4d7e5ba5744b47a854f00460dac7aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0e4d7e5ba5744b47a854f00460dac7aa, 98033857a04c4ac794f0feeead2ddc3d, cb38f9cc24a2413289605d92553b0b60; no voters:
I20260806 11:36:49.163084 29474 raft_consensus.cc:2781] T 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 P 0e4d7e5ba5744b47a854f00460dac7aa [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:36:49.163383 29474 raft_consensus.cc:686] T 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 P 0e4d7e5ba5744b47a854f00460dac7aa [term 1 LEADER]: Becoming Leader. State: Replica: 0e4d7e5ba5744b47a854f00460dac7aa, State: Running, Role: LEADER
I20260806 11:36:49.163724 29474 consensus_queue.cc:237] T 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 P 0e4d7e5ba5744b47a854f00460dac7aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb38f9cc24a2413289605d92553b0b60" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 38635 } } peers { permanent_uuid: "98033857a04c4ac794f0feeead2ddc3d" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 41935 } } peers { permanent_uuid: "0e4d7e5ba5744b47a854f00460dac7aa" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 36707 } }
I20260806 11:36:49.164372 29477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:49.164074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:46157 (local address 127.21.14.194:41109)
0806 11:36:49.164211 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:49.164280 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:46157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260806 11:36:49.166744 28878 catalog_manager.cc:5495] T 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 P 0e4d7e5ba5744b47a854f00460dac7aa reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e4d7e5ba5744b47a854f00460dac7aa (127.21.14.194). New cstate: current_term: 1 leader_uuid: "0e4d7e5ba5744b47a854f00460dac7aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb38f9cc24a2413289605d92553b0b60" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 38635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98033857a04c4ac794f0feeead2ddc3d" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 41935 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e4d7e5ba5744b47a854f00460dac7aa" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 36707 } health_report { overall_health: HEALTHY } } }
W20260806 11:36:49.173936 29362 tablet.cc:2369] T 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 P 98033857a04c4ac794f0feeead2ddc3d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 11:36:49.183007 29221 tablet.cc:2369] T 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 P 0e4d7e5ba5744b47a854f00460dac7aa: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 11:36:49.186455 29074 tablet.cc:2369] T 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 P cb38f9cc24a2413289605d92553b0b60: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 11:36:49.192302 29467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:49.188381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38692 (local address 127.21.14.194:36707)
0806 11:36:49.188540 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:49.188543 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:49.188563 (+ 20us) server_negotiation.cc:407] Connection header received
0806 11:36:49.188652 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:49.188654 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:49.188716 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:49.188833 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 11:36:49.189170 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:49.189713 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:49.190614 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:49.190939 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:49.191052 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:49.191385 (+ 333us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 11:36:49.191896 (+ 511us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 11:36:49.192009 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:49.192098 (+ 89us) server_negotiation.cc:299] Negotiation successful
0806 11:36:49.192174 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 11:36:49.195951 29019 raft_consensus.cc:1261] T 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 P cb38f9cc24a2413289605d92553b0b60 [term 1 FOLLOWER]: Refusing update from remote peer 0e4d7e5ba5744b47a854f00460dac7aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 11:36:49.196185 29303 raft_consensus.cc:1261] T 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 P 98033857a04c4ac794f0feeead2ddc3d [term 1 FOLLOWER]: Refusing update from remote peer 0e4d7e5ba5744b47a854f00460dac7aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 11:36:49.196620 29474 consensus_queue.cc:1035] T 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 P 0e4d7e5ba5744b47a854f00460dac7aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb38f9cc24a2413289605d92553b0b60" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 38635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 11:36:49.196981 29478 consensus_queue.cc:1035] T 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 P 0e4d7e5ba5744b47a854f00460dac7aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "98033857a04c4ac794f0feeead2ddc3d" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 41935 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 11:36:49.199013 29447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:49.198591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:46157 (local address 127.21.14.195:33817)
0806 11:36:49.198776 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:49.198834 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:46157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260806 11:36:49.201304 29445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:49.200955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:46157 (local address 127.21.14.193:40773)
0806 11:36:49.201127 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:49.201193 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:46157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260806 11:36:49.203172 29445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:49.202811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:46157 (local address 127.21.14.193:50877)
0806 11:36:49.203027 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:49.203084 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:46157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142}
I20260806 11:36:49.203166 29477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:49.202811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:46157 (local address 127.21.14.194:52395)
0806 11:36:49.202984 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:49.203058 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:46157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260806 11:36:49.203189 29481 mvcc.cc:205] Tried to move back new op lower bound from 7315522392859746304 to 7315522392738103296. Current Snapshot: MvccSnapshot[applied={T|T < 7315522392859746304}]
I20260806 11:36:49.206239 29477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:49.204837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:46157 (local address 127.21.14.194:33299)
0806 11:36:49.206081 (+ 1244us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:49.206157 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:46157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1150}
I20260806 11:36:49.208565 29447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:49.208251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:46157 (local address 127.21.14.195:41459)
0806 11:36:49.208381 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:49.208451 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:46157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260806 11:36:49.214996 28879 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32850:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 11:36:49.215227 28879 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32850:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 11:36:49.220135 28879 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9e5a72b344fe41a1ae5787da75d1b90f: Sending DeleteTablet for 3 replicas of tablet 9a53f1d2dcd44e0dbf11d1f68a7ff8d9
I20260806 11:36:49.221089 29145 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:36:49 UTC) from {username='oryx', principal='oryx/127.21.14.252@KRBTEST.COM'} at 127.0.0.1:38688
I20260806 11:36:49.221278 29291 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:36:49 UTC) from {username='oryx', principal='oryx/127.21.14.252@KRBTEST.COM'} at 127.0.0.1:36846
I20260806 11:36:49.221529 28997 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:36:49 UTC) from {username='oryx', principal='oryx/127.21.14.252@KRBTEST.COM'} at 127.0.0.1:57136
I20260806 11:36:49.221901 29494 tablet_replica.cc:330] T 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 P cb38f9cc24a2413289605d92553b0b60: stopping tablet replica
I20260806 11:36:49.221915 29492 tablet_replica.cc:330] T 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 P 0e4d7e5ba5744b47a854f00460dac7aa: stopping tablet replica
I20260806 11:36:49.222059 29492 raft_consensus.cc:2229] T 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 P 0e4d7e5ba5744b47a854f00460dac7aa [term 1 LEADER]: Raft consensus shutting down.
I20260806 11:36:49.222079 29494 raft_consensus.cc:2229] T 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 P cb38f9cc24a2413289605d92553b0b60 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 11:36:49.222301 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 28838
I20260806 11:36:49.222342 29494 raft_consensus.cc:2258] T 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 P cb38f9cc24a2413289605d92553b0b60 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 11:36:49.224087 29493 tablet_replica.cc:330] T 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 P 98033857a04c4ac794f0feeead2ddc3d: stopping tablet replica
I20260806 11:36:49.224248 29493 raft_consensus.cc:2229] T 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 P 98033857a04c4ac794f0feeead2ddc3d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 11:36:49.224411 29493 raft_consensus.cc:2258] T 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 P 98033857a04c4ac794f0feeead2ddc3d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 11:36:49.224653 29492 raft_consensus.cc:2258] T 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 P 0e4d7e5ba5744b47a854f00460dac7aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 11:36:49.225668 29493 ts_tablet_manager.cc:1905] T 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 P 98033857a04c4ac794f0feeead2ddc3d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 11:36:49.227409 29492 ts_tablet_manager.cc:1905] T 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 P 0e4d7e5ba5744b47a854f00460dac7aa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 11:36:49.229943 29492 ts_tablet_manager.cc:1918] T 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 P 0e4d7e5ba5744b47a854f00460dac7aa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 11:36:49.230093 29492 log.cc:1196] T 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 P 0e4d7e5ba5744b47a854f00460dac7aa: Deleting WAL directory at /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-1/wal/wals/9a53f1d2dcd44e0dbf11d1f68a7ff8d9
I20260806 11:36:49.230391 29492 ts_tablet_manager.cc:1939] T 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 P 0e4d7e5ba5744b47a854f00460dac7aa: Deleting consensus metadata
I20260806 11:36:49.231549 28862 catalog_manager.cc:4841] TS 0e4d7e5ba5744b47a854f00460dac7aa (127.21.14.194:36707): tablet 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 (table test-table [id=da38e5ef3d03490fb85f50f6a5d7b0f3]) successfully deleted
I20260806 11:36:49.232985 29493 ts_tablet_manager.cc:1918] T 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 P 98033857a04c4ac794f0feeead2ddc3d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 11:36:49.233093 29493 log.cc:1196] T 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 P 98033857a04c4ac794f0feeead2ddc3d: Deleting WAL directory at /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786016169045816-21563-0/minicluster-data/ts-2/wal/wals/9a53f1d2dcd44e0dbf11d1f68a7ff8d9
I20260806 11:36:49.233392 29493 ts_tablet_manager.cc:1939] T 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 P 98033857a04c4ac794f0feeead2ddc3d: Deleting consensus metadata
I20260806 11:36:49.235204 28864 catalog_manager.cc:4841] TS 98033857a04c4ac794f0feeead2ddc3d (127.21.14.195:41935): tablet 9a53f1d2dcd44e0dbf11d1f68a7ff8d9 (table test-table [id=da38e5ef3d03490fb85f50f6a5d7b0f3]) successfully deleted
W20260806 11:36:49.238507 29463 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:36:49.237889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.193:38635 (local address 127.0.0.1:57152)
0806 11:36:49.238261 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:49.238346 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.193:38635: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":38}
I20260806 11:36:49.238668 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 29080
W20260806 11:36:49.238773 28865 catalog_manager.cc:4584] TS cb38f9cc24a2413289605d92553b0b60 (127.21.14.193:38635): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9a53f1d2dcd44e0dbf11d1f68a7ff8d9: Network error: Client connection negotiation failed: client connection to 127.21.14.193:38635: connect: Connection reset by peer (error 104)
I20260806 11:36:49.248756 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 29225
I20260806 11:36:49.258543 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 28537
I20260806 11:36:49.266983 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 28645
2026-08-06T11:36:49Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (7419 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 11:36:49 dist-test-slave-7c35 krb5kdc[29497](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 11:36:49 dist-test-slave-7c35 krb5kdc[29497](info): set up 2 sockets
Aug 06 11:36:49 dist-test-slave-7c35 krb5kdc[29497](info): commencing operation
krb5kdc: starting...
W20260806 11:36:50.371243 21563 mini_kdc.cc:121] Time spent starting KDC: real 1.076s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Aug 06 11:36:50 dist-test-slave-7c35 krb5kdc[29497](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016210, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T11:36:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T11:36:50Z Disabled control of system clock
WARNING: no policy specified for oryx/127.21.14.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.14.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.14.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/krb5kdc/oryx_127.21.14.254.keytab.
Entry for principal oryx/127.21.14.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/krb5kdc/oryx_127.21.14.254.keytab.
I20260806 11:36:50.532876 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:35125
--webserver_interface=127.21.14.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/krb5kdc/oryx_127.21.14.254.keytab
--principal=oryx/127.21.14.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:44295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.14.254:35125
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:36:50.661391 29511 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:36:50.661794 29511 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:36:50.661887 29511 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:36:50.666230 29511 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 11:36:50.666369 29511 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 11:36:50.666404 29511 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:36:50.666447 29511 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:36:50.666494 29511 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 11:36:50.666528 29511 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 11:36:50.672497 29511 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:44295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.14.254:35125
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/krb5kdc/oryx_127.21.14.254.keytab
--principal=oryx/127.21.14.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:35125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.14.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29511
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:36:50.673812 29511 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:36:50.675032 29511 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 11:36:50.685334 29511 server_base.cc:1034] running on GCE node
W20260806 11:36:50.685386 29516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:50.685401 29517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:50.685371 29519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:50.686512 29511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:36:50.688068 29511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:36:50.689321 29511 hybrid_clock.cc:648] HybridClock initialized: now 1786016210689278 us; error 61 us; skew 500 ppm
Aug 06 11:36:50 dist-test-slave-7c35 krb5kdc[29497](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016210, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.14.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:36:50.693020 29511 init.cc:377] Logged in from keytab as oryx/127.21.14.254@KRBTEST.COM (short username oryx)
I20260806 11:36:50.694363 29511 webserver.cc:460] Webserver started at http://127.21.14.254:42785/ using document root <none> and password file <none>
I20260806 11:36:50.695340 29511 fs_manager.cc:359] Metadata directory not provided
I20260806 11:36:50.695470 29511 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:36:50.695776 29511 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:36:50.698431 29511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/master-0/data/instance:
uuid: "ed204855a8dd4cd686fd550178b9563e"
format_stamp: "Formatted at 2026-08-06 11:36:50 on dist-test-slave-7c35"
server_key: "b81a315de0f55c9b0c9931fa26828492"
server_key_iv: "1b618bcbe948d750152e492d71803429"
server_key_version: "encryptionkey@0"
I20260806 11:36:50.699205 29511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/master-0/wal/instance:
uuid: "ed204855a8dd4cd686fd550178b9563e"
format_stamp: "Formatted at 2026-08-06 11:36:50 on dist-test-slave-7c35"
server_key: "b81a315de0f55c9b0c9931fa26828492"
server_key_iv: "1b618bcbe948d750152e492d71803429"
server_key_version: "encryptionkey@0"
I20260806 11:36:50.704383 29511 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260806 11:36:50.707599 29526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:50.709190 29511 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 11:36:50.709394 29511 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/master-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/master-0/wal
uuid: "ed204855a8dd4cd686fd550178b9563e"
format_stamp: "Formatted at 2026-08-06 11:36:50 on dist-test-slave-7c35"
server_key: "b81a315de0f55c9b0c9931fa26828492"
server_key_iv: "1b618bcbe948d750152e492d71803429"
server_key_version: "encryptionkey@0"
I20260806 11:36:50.709542 29511 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:36:50.732831 29511 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:36:50.736433 29511 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:36:50.736677 29511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:36:50.745906 29511 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.254:35125
I20260806 11:36:50.745930 29592 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.254:35125 every 8 connection(s)
I20260806 11:36:50.747151 29511 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
I20260806 11:36:50.750519 29593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:50.752748 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 29511
I20260806 11:36:50.752933 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/master-0/wal/instance
I20260806 11:36:50.753242 21563 external_mini_cluster.cc:1428] Setting key 92301b77cadf76b126b31bd00ca8aeb8
I20260806 11:36:50.758438 29593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed204855a8dd4cd686fd550178b9563e: Bootstrap starting.
I20260806 11:36:50.761497 29593 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ed204855a8dd4cd686fd550178b9563e: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:50.762630 29593 log.cc:824] T 00000000000000000000000000000000 P ed204855a8dd4cd686fd550178b9563e: Log is configured to *not* fsync() on all Append() calls
Aug 06 11:36:50 dist-test-slave-7c35 krb5kdc[29497](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016210, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM
I20260806 11:36:50.765376 29593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed204855a8dd4cd686fd550178b9563e: No bootstrap required, opened a new log
I20260806 11:36:50.768872 29593 raft_consensus.cc:348] T 00000000000000000000000000000000 P ed204855a8dd4cd686fd550178b9563e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed204855a8dd4cd686fd550178b9563e" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 35125 } }
I20260806 11:36:50.769223 29593 raft_consensus.cc:374] T 00000000000000000000000000000000 P ed204855a8dd4cd686fd550178b9563e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:50.769309 29593 raft_consensus.cc:729] T 00000000000000000000000000000000 P ed204855a8dd4cd686fd550178b9563e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed204855a8dd4cd686fd550178b9563e, State: Initialized, Role: FOLLOWER
I20260806 11:36:50.769868 29593 consensus_queue.cc:260] T 00000000000000000000000000000000 P ed204855a8dd4cd686fd550178b9563e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed204855a8dd4cd686fd550178b9563e" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 35125 } }
I20260806 11:36:50.770056 29593 raft_consensus.cc:388] T 00000000000000000000000000000000 P ed204855a8dd4cd686fd550178b9563e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 11:36:50.770174 29593 raft_consensus.cc:482] T 00000000000000000000000000000000 P ed204855a8dd4cd686fd550178b9563e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 11:36:50.770310 29593 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ed204855a8dd4cd686fd550178b9563e [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:36:50.770808 29596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:50.754913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60554 (local address 127.21.14.254:35125)
0806 11:36:50.755421 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:50.755440 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:50.755485 (+ 45us) server_negotiation.cc:407] Connection header received
0806 11:36:50.756237 (+ 752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:50.756258 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:50.756652 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:50.756999 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:50.758065 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:50.759298 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:50.760552 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:50.761122 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:50.761262 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:50.764465 (+ 3203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:50.764471 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:50.764489 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:50.764519 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:50.767298 (+ 2779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:50.767849 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:50.767854 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:50.767859 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:50.767987 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:50.768308 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:50.768311 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:50.768313 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:50.768788 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:50.769014 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:50.769386 (+ 372us) server_negotiation.cc:299] Negotiation successful
0806 11:36:50.769759 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":155,"threads_started":1}
I20260806 11:36:50.771574 29593 raft_consensus.cc:504] T 00000000000000000000000000000000 P ed204855a8dd4cd686fd550178b9563e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed204855a8dd4cd686fd550178b9563e" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 35125 } }
I20260806 11:36:50.771984 29593 leader_election.cc:302] T 00000000000000000000000000000000 P ed204855a8dd4cd686fd550178b9563e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ed204855a8dd4cd686fd550178b9563e; no voters:
I20260806 11:36:50.772393 29593 leader_election.cc:288] T 00000000000000000000000000000000 P ed204855a8dd4cd686fd550178b9563e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 11:36:50.772496 29598 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ed204855a8dd4cd686fd550178b9563e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:36:50.772755 29598 raft_consensus.cc:686] T 00000000000000000000000000000000 P ed204855a8dd4cd686fd550178b9563e [term 1 LEADER]: Becoming Leader. State: Replica: ed204855a8dd4cd686fd550178b9563e, State: Running, Role: LEADER
I20260806 11:36:50.773160 29598 consensus_queue.cc:237] T 00000000000000000000000000000000 P ed204855a8dd4cd686fd550178b9563e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed204855a8dd4cd686fd550178b9563e" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 35125 } }
I20260806 11:36:50.773844 29593 sys_catalog.cc:564] T 00000000000000000000000000000000 P ed204855a8dd4cd686fd550178b9563e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 11:36:50.774803 29598 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed204855a8dd4cd686fd550178b9563e [sys.catalog]: SysCatalogTable state changed. Reason: New leader ed204855a8dd4cd686fd550178b9563e. Latest consensus state: current_term: 1 leader_uuid: "ed204855a8dd4cd686fd550178b9563e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed204855a8dd4cd686fd550178b9563e" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 35125 } } }
I20260806 11:36:50.775081 29598 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed204855a8dd4cd686fd550178b9563e [sys.catalog]: This master's current role is: LEADER
I20260806 11:36:50.775383 29599 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed204855a8dd4cd686fd550178b9563e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ed204855a8dd4cd686fd550178b9563e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed204855a8dd4cd686fd550178b9563e" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 35125 } } }
I20260806 11:36:50.775542 29599 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed204855a8dd4cd686fd550178b9563e [sys.catalog]: This master's current role is: LEADER
I20260806 11:36:50.775625 29607 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 11:36:50.779287 29607 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 11:36:50.785797 29607 catalog_manager.cc:1347] Generated new cluster ID: 5a8886da1b7f40e6b83719821fecd62f
I20260806 11:36:50.785935 29607 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 11:36:50.801570 29607 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 11:36:50.802731 29607 catalog_manager.cc:1504] Loading token signing keys...
I20260806 11:36:50.817513 29607 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ed204855a8dd4cd686fd550178b9563e: Generated new TSK 0
I20260806 11:36:50.818428 29607 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.21.14.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.14.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.14.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/krb5kdc/oryx_127.21.14.193.keytab.
Entry for principal oryx/127.21.14.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/krb5kdc/oryx_127.21.14.193.keytab.
I20260806 11:36:50.863490 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.193:0
--local_ip_for_outbound_sockets=127.21.14.193
--webserver_interface=127.21.14.193
--webserver_port=0
--tserver_master_addrs=127.21.14.254:35125
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/krb5kdc/oryx_127.21.14.193.keytab
--principal=oryx/127.21.14.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:44295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:36:50.988039 29620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:36:50.988511 29620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:36:50.988583 29620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:36:50.993412 29620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:36:50.993502 29620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:36:50.993547 29620 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 11:36:50.993582 29620 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.193
I20260806 11:36:50.999406 29620 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:44295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/krb5kdc/oryx_127.21.14.193.keytab
--principal=oryx/127.21.14.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.14.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.14.254:35125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29620
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.193
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:36:51.000605 29620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:36:51.001792 29620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:36:51.012988 29626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:51.013072 29620 server_base.cc:1034] running on GCE node
W20260806 11:36:51.012938 29625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:51.012938 29628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:51.013823 29620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:36:51.014731 29620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:36:51.015955 29620 hybrid_clock.cc:648] HybridClock initialized: now 1786016211015930 us; error 42 us; skew 500 ppm
Aug 06 11:36:51 dist-test-slave-7c35 krb5kdc[29497](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016211, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.14.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:36:51.024449 29620 init.cc:377] Logged in from keytab as oryx/127.21.14.193@KRBTEST.COM (short username oryx)
I20260806 11:36:51.025565 29620 webserver.cc:460] Webserver started at http://127.21.14.193:46347/ using document root <none> and password file <none>
I20260806 11:36:51.026176 29620 fs_manager.cc:359] Metadata directory not provided
I20260806 11:36:51.026242 29620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:36:51.026423 29620 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:36:51.028828 29620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/ts-0/data/instance:
uuid: "741edc4087df42139936dace73d5f68a"
format_stamp: "Formatted at 2026-08-06 11:36:51 on dist-test-slave-7c35"
server_key: "8d8f3a2ef1ae0cc87f75c01600d8e861"
server_key_iv: "38ddf24cdb27654a3bc460483a6533b4"
server_key_version: "encryptionkey@0"
I20260806 11:36:51.029421 29620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/ts-0/wal/instance:
uuid: "741edc4087df42139936dace73d5f68a"
format_stamp: "Formatted at 2026-08-06 11:36:51 on dist-test-slave-7c35"
server_key: "8d8f3a2ef1ae0cc87f75c01600d8e861"
server_key_iv: "38ddf24cdb27654a3bc460483a6533b4"
server_key_version: "encryptionkey@0"
I20260806 11:36:51.034247 29620 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260806 11:36:51.037413 29636 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:51.039080 29620 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 11:36:51.039248 29620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/ts-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/ts-0/wal
uuid: "741edc4087df42139936dace73d5f68a"
format_stamp: "Formatted at 2026-08-06 11:36:51 on dist-test-slave-7c35"
server_key: "8d8f3a2ef1ae0cc87f75c01600d8e861"
server_key_iv: "38ddf24cdb27654a3bc460483a6533b4"
server_key_version: "encryptionkey@0"
I20260806 11:36:51.039387 29620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:36:51.061542 29620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:36:51.065745 29620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:36:51.066032 29620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:36:51.067847 29620 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:36:51.067919 29620 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:51.068154 29620 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:36:51.068221 29620 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 11:36:51 dist-test-slave-7c35 krb5kdc[29497](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016211, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.14.193@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM
I20260806 11:36:51.080600 29620 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.193:40189
I20260806 11:36:51.081808 29620 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
I20260806 11:36:51.082768 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 29620
I20260806 11:36:51.082788 29758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.193:40189 every 8 connection(s)
I20260806 11:36:51.082950 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/ts-0/wal/instance
I20260806 11:36:51.083236 21563 external_mini_cluster.cc:1428] Setting key a7a51004db8426e2555fea3c2af2c24b
I20260806 11:36:51.086062 29596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:51.068887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:60389 (local address 127.21.14.254:35125)
0806 11:36:51.069034 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:51.069045 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:51.070358 (+ 1313us) server_negotiation.cc:407] Connection header received
0806 11:36:51.071662 (+ 1304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:51.071666 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:51.071746 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:51.071815 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:51.074027 (+ 2212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:51.075069 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:51.077345 (+ 2276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:51.077627 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:51.077730 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:51.081306 (+ 3576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:51.081310 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:51.081313 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:51.081343 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:51.083179 (+ 1836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:51.084113 (+ 934us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:51.084117 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:51.084119 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:51.084223 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:51.084639 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:51.084642 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:51.084644 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:51.084906 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:51.085021 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:51.085587 (+ 566us) server_negotiation.cc:299] Negotiation successful
0806 11:36:51.085778 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 11:36:51.086794 29646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:51.069528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:35125 (local address 127.21.14.193:60389)
0806 11:36:51.070157 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:51.070214 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:51.071315 (+ 1101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:51.072371 (+ 1056us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:51.072385 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:51.072906 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:51.073736 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:51.073770 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:51.075309 (+ 1539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:51.075313 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:51.077130 (+ 1817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:51.077148 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:51.077841 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:51.077847 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:51.078137 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:51.079024 (+ 887us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:51.079050 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:51.081042 (+ 1992us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:51.083345 (+ 2303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:51.083352 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:51.083370 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:51.083775 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:51.084338 (+ 563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:51.084340 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:51.084342 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:51.084496 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:51.085019 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:51.085031 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:51.085304 (+ 273us) client_negotiation.cc:769] Sending connection context
0806 11:36:51.085483 (+ 179us) client_negotiation.cc:240] Negotiation successful
0806 11:36:51.085790 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":162,"threads_started":1}
I20260806 11:36:51.088303 29759 heartbeater.cc:347] Connected to a master server at 127.21.14.254:35125
I20260806 11:36:51.088588 29759 heartbeater.cc:464] Registering TS with master...
I20260806 11:36:51.089186 29759 heartbeater.cc:511] Master 127.21.14.254:35125 requested a full tablet report, sending...
I20260806 11:36:51.091122 29542 ts_manager.cc:194] Registered new tserver with Master: 741edc4087df42139936dace73d5f68a (127.21.14.193:40189)
I20260806 11:36:51.092809 29542 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.14.193@KRBTEST.COM'} at 127.21.14.193:60389
WARNING: no policy specified for oryx/127.21.14.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.14.194@KRBTEST.COM" created.
I20260806 11:36:51.105698 29766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:51.095393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:35125 (local address 127.21.14.193:49963)
0806 11:36:51.095985 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:51.095999 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:51.096169 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:51.096517 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:51.096520 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:51.096720 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:51.096892 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:51.096908 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:51.097896 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:51.097901 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:51.098749 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:51.098763 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:51.099374 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:51.099379 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:51.099494 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:51.100208 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:51.100222 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:51.100622 (+ 400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:51.103322 (+ 2700us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:51.103336 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:51.103338 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:51.103598 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:51.104135 (+ 537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:51.104139 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:51.104142 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:51.104252 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:51.105009 (+ 757us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:51.105013 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:51.105137 (+ 124us) client_negotiation.cc:769] Sending connection context
0806 11:36:51.105269 (+ 132us) client_negotiation.cc:240] Negotiation successful
0806 11:36:51.105490 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":149,"threads_started":1}
I20260806 11:36:51.105693 29596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:51.095500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:49963 (local address 127.21.14.254:35125)
0806 11:36:51.095653 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:51.095658 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:51.096257 (+ 599us) server_negotiation.cc:407] Connection header received
0806 11:36:51.096317 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:51.096319 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:51.096366 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:51.096448 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:51.097040 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:51.097720 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:51.098901 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:51.099158 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:51.099377 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:51.100830 (+ 1453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:51.100833 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:51.100835 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:51.100864 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:51.103110 (+ 2246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:51.103879 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:51.103883 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:51.103885 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:51.103989 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:51.104373 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:51.104379 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:51.104385 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:51.104852 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:51.105008 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:51.105334 (+ 326us) server_negotiation.cc:299] Negotiation successful
0806 11:36:51.105486 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.14.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/krb5kdc/oryx_127.21.14.194.keytab.
Entry for principal oryx/127.21.14.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/krb5kdc/oryx_127.21.14.194.keytab.
I20260806 11:36:51.126657 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.194:0
--local_ip_for_outbound_sockets=127.21.14.194
--webserver_interface=127.21.14.194
--webserver_port=0
--tserver_master_addrs=127.21.14.254:35125
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/krb5kdc/oryx_127.21.14.194.keytab
--principal=oryx/127.21.14.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:44295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:36:51.255534 29768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:36:51.255847 29768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:36:51.255893 29768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:36:51.260146 29768 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:36:51.260231 29768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:36:51.260273 29768 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 11:36:51.260308 29768 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.194
I20260806 11:36:51.266100 29768 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:44295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/krb5kdc/oryx_127.21.14.194.keytab
--principal=oryx/127.21.14.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.14.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.14.254:35125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29768
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.194
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:36:51.267359 29768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:36:51.268530 29768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:36:51.279975 29774 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:51.279964 29773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:51.279994 29776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:51.280539 29768 server_base.cc:1034] running on GCE node
I20260806 11:36:51.281088 29768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:36:51.281953 29768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:36:51.283171 29768 hybrid_clock.cc:648] HybridClock initialized: now 1786016211283147 us; error 38 us; skew 500 ppm
Aug 06 11:36:51 dist-test-slave-7c35 krb5kdc[29497](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016211, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.14.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:36:51.286682 29768 init.cc:377] Logged in from keytab as oryx/127.21.14.194@KRBTEST.COM (short username oryx)
I20260806 11:36:51.287873 29768 webserver.cc:460] Webserver started at http://127.21.14.194:40117/ using document root <none> and password file <none>
I20260806 11:36:51.288632 29768 fs_manager.cc:359] Metadata directory not provided
I20260806 11:36:51.288710 29768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:36:51.288959 29768 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:36:51.291934 29768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/ts-1/data/instance:
uuid: "92712864d28043d5aa2771fbc08fc004"
format_stamp: "Formatted at 2026-08-06 11:36:51 on dist-test-slave-7c35"
server_key: "89492bc5202dad48de2d21e35e91e52e"
server_key_iv: "5df47ad575c07723c2e2cfdb054b596a"
server_key_version: "encryptionkey@0"
I20260806 11:36:51.292651 29768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/ts-1/wal/instance:
uuid: "92712864d28043d5aa2771fbc08fc004"
format_stamp: "Formatted at 2026-08-06 11:36:51 on dist-test-slave-7c35"
server_key: "89492bc5202dad48de2d21e35e91e52e"
server_key_iv: "5df47ad575c07723c2e2cfdb054b596a"
server_key_version: "encryptionkey@0"
I20260806 11:36:51.297382 29768 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 11:36:51.300493 29783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:51.302059 29768 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 11:36:51.302193 29768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/ts-1/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/ts-1/wal
uuid: "92712864d28043d5aa2771fbc08fc004"
format_stamp: "Formatted at 2026-08-06 11:36:51 on dist-test-slave-7c35"
server_key: "89492bc5202dad48de2d21e35e91e52e"
server_key_iv: "5df47ad575c07723c2e2cfdb054b596a"
server_key_version: "encryptionkey@0"
I20260806 11:36:51.302314 29768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:36:51.316956 29768 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:36:51.321549 29768 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:36:51.321969 29768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:36:51.323786 29768 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:36:51.323896 29768 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:51.323971 29768 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:36:51.324018 29768 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 11:36:51 dist-test-slave-7c35 krb5kdc[29497](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016211, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.14.194@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM
I20260806 11:36:51.336740 29768 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.194:37297
I20260806 11:36:51.336819 29909 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.194:37297 every 8 connection(s)
I20260806 11:36:51.338034 29768 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
I20260806 11:36:51.342700 29596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:51.324917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:56289 (local address 127.21.14.254:35125)
0806 11:36:51.325075 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:51.325079 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:51.326443 (+ 1364us) server_negotiation.cc:407] Connection header received
0806 11:36:51.327844 (+ 1401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:51.327848 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:51.327920 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:51.328032 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:51.329921 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:51.330831 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:51.333117 (+ 2286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:51.333370 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:51.333486 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:51.337560 (+ 4074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:51.337567 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:51.337574 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:51.337619 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:51.339866 (+ 2247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:51.340743 (+ 877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:51.340749 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:51.340756 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:51.340856 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:51.341297 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:51.341301 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:51.341303 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:51.341575 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:51.341706 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:51.342345 (+ 639us) server_negotiation.cc:299] Negotiation successful
0806 11:36:51.342513 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 11:36:51.344081 29794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:51.325605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:35125 (local address 127.21.14.194:56289)
0806 11:36:51.326233 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:51.326297 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:51.327574 (+ 1277us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:51.328272 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:51.328289 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:51.328791 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:51.329704 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:51.329732 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:51.331063 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:51.331071 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:51.332918 (+ 1847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:51.332936 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:51.333608 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:51.333613 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:51.333909 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:51.335167 (+ 1258us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:51.335196 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:51.337254 (+ 2058us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:51.340179 (+ 2925us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:51.340187 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:51.340208 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:51.340592 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:51.340979 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:51.340982 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:51.340984 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:51.341175 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:51.341707 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:51.341717 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:51.342048 (+ 331us) client_negotiation.cc:769] Sending connection context
0806 11:36:51.342280 (+ 232us) client_negotiation.cc:240] Negotiation successful
0806 11:36:51.342710 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":153,"threads_started":1}
I20260806 11:36:51.346024 29910 heartbeater.cc:347] Connected to a master server at 127.21.14.254:35125
I20260806 11:36:51.346267 29910 heartbeater.cc:464] Registering TS with master...
I20260806 11:36:51.346382 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 29768
I20260806 11:36:51.346521 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/ts-1/wal/instance
I20260806 11:36:51.346935 21563 external_mini_cluster.cc:1428] Setting key a36301ef0a078762f4070bc974bbcf04
I20260806 11:36:51.347090 29910 heartbeater.cc:511] Master 127.21.14.254:35125 requested a full tablet report, sending...
I20260806 11:36:51.348348 29543 ts_manager.cc:194] Registered new tserver with Master: 92712864d28043d5aa2771fbc08fc004 (127.21.14.194:37297)
I20260806 11:36:51.349241 29543 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.14.194@KRBTEST.COM'} at 127.21.14.194:56289
I20260806 11:36:51.360404 29596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:51.349196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:60539 (local address 127.21.14.254:35125)
0806 11:36:51.349350 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:51.349357 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:51.349649 (+ 292us) server_negotiation.cc:407] Connection header received
0806 11:36:51.349780 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:51.349785 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:51.349849 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:51.349955 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:51.350518 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:51.351162 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:51.353033 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:51.353317 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:51.353448 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:51.355782 (+ 2334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:51.355786 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:51.355789 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:51.355820 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:51.357511 (+ 1691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:51.358448 (+ 937us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:51.358452 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:51.358454 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:51.358573 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:51.359215 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:51.359219 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:51.359220 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:51.359509 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:51.359616 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:51.360114 (+ 498us) server_negotiation.cc:299] Negotiation successful
0806 11:36:51.360254 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 11:36:51.360463 29916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:51.349092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:35125 (local address 127.21.14.194:60539)
0806 11:36:51.349520 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:51.349551 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:51.349675 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:51.350000 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:51.350003 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:51.350259 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:51.350382 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:51.350389 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:51.351299 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:51.351303 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:51.352680 (+ 1377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:51.352697 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:51.353481 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:51.353487 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:51.353619 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:51.354622 (+ 1003us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:51.354725 (+ 103us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:51.355524 (+ 799us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:51.357694 (+ 2170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:51.357699 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:51.357701 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:51.358248 (+ 547us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:51.358707 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:51.358712 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:51.358714 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:51.358988 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:51.359644 (+ 656us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:51.359647 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:51.359787 (+ 140us) client_negotiation.cc:769] Sending connection context
0806 11:36:51.359933 (+ 146us) client_negotiation.cc:240] Negotiation successful
0806 11:36:51.360159 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"mutex_wait_us":5,"thread_start_us":147,"threads_started":1}
WARNING: no policy specified for oryx/127.21.14.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.14.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.14.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/krb5kdc/oryx_127.21.14.195.keytab.
Entry for principal oryx/127.21.14.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/krb5kdc/oryx_127.21.14.195.keytab.
I20260806 11:36:51.388661 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.195:0
--local_ip_for_outbound_sockets=127.21.14.195
--webserver_interface=127.21.14.195
--webserver_port=0
--tserver_master_addrs=127.21.14.254:35125
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/krb5kdc/oryx_127.21.14.195.keytab
--principal=oryx/127.21.14.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:44295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:36:51.520423 29919 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:36:51.520778 29919 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:36:51.520828 29919 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:36:51.525357 29919 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:36:51.525450 29919 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:36:51.525493 29919 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 11:36:51.525566 29919 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.195
I20260806 11:36:51.531538 29919 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:44295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/krb5kdc/oryx_127.21.14.195.keytab
--principal=oryx/127.21.14.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.14.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.14.254:35125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29919
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.195
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:36:51.532837 29919 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:36:51.534056 29919 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:36:51.546185 29926 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:51.546187 29927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:51.546185 29929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:51.546242 29919 server_base.cc:1034] running on GCE node
I20260806 11:36:51.547195 29919 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:36:51.547964 29919 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:36:51.549194 29919 hybrid_clock.cc:648] HybridClock initialized: now 1786016211549177 us; error 49 us; skew 500 ppm
Aug 06 11:36:51 dist-test-slave-7c35 krb5kdc[29497](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016211, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.14.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:36:51.552601 29919 init.cc:377] Logged in from keytab as oryx/127.21.14.195@KRBTEST.COM (short username oryx)
I20260806 11:36:51.553702 29919 webserver.cc:460] Webserver started at http://127.21.14.195:35245/ using document root <none> and password file <none>
I20260806 11:36:51.554302 29919 fs_manager.cc:359] Metadata directory not provided
I20260806 11:36:51.554368 29919 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:36:51.554569 29919 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:36:51.556926 29919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/ts-2/data/instance:
uuid: "f1d856374ce44857a183737b5e2b03ba"
format_stamp: "Formatted at 2026-08-06 11:36:51 on dist-test-slave-7c35"
server_key: "5568eb0deecf9adbd2685e0c6029aab4"
server_key_iv: "0a6a56e8b6bdeabaa2bb9b7c53d72a4c"
server_key_version: "encryptionkey@0"
I20260806 11:36:51.557596 29919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/ts-2/wal/instance:
uuid: "f1d856374ce44857a183737b5e2b03ba"
format_stamp: "Formatted at 2026-08-06 11:36:51 on dist-test-slave-7c35"
server_key: "5568eb0deecf9adbd2685e0c6029aab4"
server_key_iv: "0a6a56e8b6bdeabaa2bb9b7c53d72a4c"
server_key_version: "encryptionkey@0"
I20260806 11:36:51.562012 29919 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 11:36:51.565140 29937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:51.566690 29919 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 11:36:51.566823 29919 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/ts-2/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/ts-2/wal
uuid: "f1d856374ce44857a183737b5e2b03ba"
format_stamp: "Formatted at 2026-08-06 11:36:51 on dist-test-slave-7c35"
server_key: "5568eb0deecf9adbd2685e0c6029aab4"
server_key_iv: "0a6a56e8b6bdeabaa2bb9b7c53d72a4c"
server_key_version: "encryptionkey@0"
I20260806 11:36:51.567056 29919 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:36:51.580785 29919 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:36:51.584586 29919 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:36:51.584774 29919 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:36:51.586408 29919 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:36:51.586483 29919 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:51.586539 29919 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:36:51.586601 29919 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 11:36:51 dist-test-slave-7c35 krb5kdc[29497](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016211, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.14.195@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM
I20260806 11:36:51.599066 29919 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.195:38073
I20260806 11:36:51.599367 30068 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.195:38073 every 8 connection(s)
I20260806 11:36:51.600224 29919 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
I20260806 11:36:51.602418 29596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:51.587383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:57089 (local address 127.21.14.254:35125)
0806 11:36:51.587546 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:51.587550 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:51.588808 (+ 1258us) server_negotiation.cc:407] Connection header received
0806 11:36:51.590184 (+ 1376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:51.590188 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:51.590273 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:51.590341 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:51.592246 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:51.592843 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:51.594217 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:51.594556 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:51.594699 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:51.597901 (+ 3202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:51.597905 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:51.597907 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:51.597940 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:51.599853 (+ 1913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:51.600513 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:51.600520 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:51.600526 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:51.600620 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:51.601095 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:51.601101 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:51.601105 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:51.601352 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:51.601483 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:51.602057 (+ 574us) server_negotiation.cc:299] Negotiation successful
0806 11:36:51.602228 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 11:36:51.603583 29948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:51.588032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:35125 (local address 127.21.14.195:57089)
0806 11:36:51.588625 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:51.588690 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:51.589870 (+ 1180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:51.590788 (+ 918us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:51.590796 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:51.591361 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:51.592066 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:51.592084 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:51.593008 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:51.593012 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:51.594065 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:51.594078 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:51.594761 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:51.594766 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:51.595033 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:51.595831 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:51.595850 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:51.597656 (+ 1806us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:51.600030 (+ 2374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:51.600037 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:51.600055 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:51.600399 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:51.600725 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:51.600729 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:51.600731 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:51.600901 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:51.601481 (+ 580us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:51.601490 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:51.601775 (+ 285us) client_negotiation.cc:769] Sending connection context
0806 11:36:51.601995 (+ 220us) client_negotiation.cc:240] Negotiation successful
0806 11:36:51.602386 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":160,"threads_started":1}
I20260806 11:36:51.605151 30069 heartbeater.cc:347] Connected to a master server at 127.21.14.254:35125
I20260806 11:36:51.605369 30069 heartbeater.cc:464] Registering TS with master...
I20260806 11:36:51.606125 30069 heartbeater.cc:511] Master 127.21.14.254:35125 requested a full tablet report, sending...
I20260806 11:36:51.607889 29542 ts_manager.cc:194] Registered new tserver with Master: f1d856374ce44857a183737b5e2b03ba (127.21.14.195:38073)
I20260806 11:36:51.609011 29542 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.14.195@KRBTEST.COM'} at 127.21.14.195:57089
I20260806 11:36:51.609041 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 29919
I20260806 11:36:51.609193 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/ts-2/wal/instance
I20260806 11:36:51.609609 21563 external_mini_cluster.cc:1428] Setting key 7f42c127c4e5b0f1f84274264a03809e
I20260806 11:36:51.612378 21563 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 11:36:51.621107 29596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:51.608667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:59571 (local address 127.21.14.254:35125)
0806 11:36:51.608814 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:51.608820 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:51.609641 (+ 821us) server_negotiation.cc:407] Connection header received
0806 11:36:51.609817 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:51.609820 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:51.609872 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:51.609958 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:51.610781 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:51.611567 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:51.612903 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:51.613150 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:51.613250 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:51.615357 (+ 2107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:51.615364 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:51.615368 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:51.615450 (+ 82us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:51.618282 (+ 2832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:51.619353 (+ 1071us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:51.619360 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:51.619365 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:51.619522 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:51.619948 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:51.619954 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:51.619956 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:51.620251 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:51.620361 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:51.620792 (+ 431us) server_negotiation.cc:299] Negotiation successful
0806 11:36:51.620934 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 11:36:51.621119 30076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:51.609052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:35125 (local address 127.21.14.195:59571)
0806 11:36:51.609529 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:51.609552 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:51.609693 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:51.610098 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:51.610104 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:51.610383 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:51.610533 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:51.610541 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:51.611738 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:51.611743 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:51.612738 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:51.612751 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:51.613294 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:51.613301 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:51.613449 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:51.614193 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:51.614214 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:51.615020 (+ 806us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:51.618488 (+ 3468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:51.618495 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:51.618499 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:51.619012 (+ 513us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:51.619681 (+ 669us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:51.619688 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:51.619692 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:51.619814 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:51.620430 (+ 616us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:51.620437 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:51.620571 (+ 134us) client_negotiation.cc:769] Sending connection context
0806 11:36:51.620719 (+ 148us) client_negotiation.cc:240] Negotiation successful
0806 11:36:51.620917 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":189,"threads_started":1}
Aug 06 11:36:51 dist-test-slave-7c35 krb5kdc[29497](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016211, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 11:36:51 dist-test-slave-7c35 krb5kdc[29497](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 11:36:51 dist-test-slave-7c35 krb5kdc[29497](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
W20260806 11:36:51.644898 29596 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:36:51.637215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60568 (local address 127.21.14.254:35125)
0806 11:36:51.637377 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:51.637381 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:51.637396 (+ 15us) server_negotiation.cc:407] Connection header received
0806 11:36:51.637446 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:51.637449 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:51.637509 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:51.637635 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:51.638560 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:51.639232 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:51.640221 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:51.640471 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:51.640598 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:51.644691 (+ 4093us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60568: BlockingRecv error: recv got EOF from 127.0.0.1:60568 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 06 11:36:51 dist-test-slave-7c35 krb5kdc[29497](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016211, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM
I20260806 11:36:51.656265 29596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:51.646557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60574 (local address 127.21.14.254:35125)
0806 11:36:51.646733 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:51.646737 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:51.646929 (+ 192us) server_negotiation.cc:407] Connection header received
0806 11:36:51.647021 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:51.647024 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:51.647072 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:51.647140 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:51.647935 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:51.648494 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:51.649335 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:51.649586 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:51.649673 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:51.652730 (+ 3057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:51.652733 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:51.652736 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:51.652797 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:51.654263 (+ 1466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:51.654801 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:51.654805 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:51.654807 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:51.654972 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:51.655289 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:51.655292 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:51.655294 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:51.655519 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:51.655681 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:51.655939 (+ 258us) server_negotiation.cc:299] Negotiation successful
0806 11:36:51.656079 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 11:36:51.659792 29542 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60574:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 11:36:51.663128 29542 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 11:36:51.680428 30097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:51.673536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.193:40189 (local address 127.0.0.1:34180)
0806 11:36:51.674304 (+ 768us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:51.674369 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:51.674540 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:51.675543 (+ 1003us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:51.675550 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:51.675591 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:36:51.675943 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:51.675952 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:51.677549 (+ 1597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:51.677552 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:51.678755 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:51.678770 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:51.679767 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:51.679771 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:51.679893 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:51.679935 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 11:36:51.680066 (+ 131us) client_negotiation.cc:240] Negotiation successful
0806 11:36:51.680228 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":674,"mutex_wait_us":248,"spinlock_wait_cycles":31872,"thread_start_us":166,"threads_started":1}
I20260806 11:36:51.680720 30093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:51.673855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.195:38073 (local address 127.0.0.1:46500)
0806 11:36:51.675360 (+ 1505us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:51.675374 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:51.675472 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:51.676078 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:51.676081 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:51.676100 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:36:51.676227 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:51.676234 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:51.677224 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:51.677228 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:51.678908 (+ 1680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:51.678921 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:51.680315 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:51.680319 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:51.680431 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:51.680457 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 11:36:51.680530 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 11:36:51.680602 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1443,"mutex_wait_us":155,"thread_start_us":106,"threads_started":1}
I20260806 11:36:51.680986 30098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:51.674398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34180 (local address 127.21.14.193:40189)
0806 11:36:51.674951 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:51.674961 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:51.674989 (+ 28us) server_negotiation.cc:407] Connection header received
0806 11:36:51.675109 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:51.675113 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:51.675347 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:51.675701 (+ 354us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:36:51.676105 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:51.677364 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:51.678953 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:51.679619 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:51.679762 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:51.679951 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:51.680698 (+ 747us) server_negotiation.cc:299] Negotiation successful
0806 11:36:51.680844 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":77,"threads_started":1}
I20260806 11:36:51.680559 30094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:51.673488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.194:37297 (local address 127.0.0.1:60302)
0806 11:36:51.674304 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:51.674369 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:51.674540 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:51.675543 (+ 1003us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:51.675550 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:51.675579 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:36:51.675777 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:51.675785 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:51.677492 (+ 1707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:51.677496 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:51.678911 (+ 1415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:51.678921 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:51.680101 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:51.680105 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:51.680200 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:51.680221 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 11:36:51.680291 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 11:36:51.680351 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"mutex_wait_us":53,"spinlock_wait_cycles":19328,"thread_start_us":87,"threads_started":1}
I20260806 11:36:51.681030 30095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:51.673686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60302 (local address 127.21.14.194:37297)
0806 11:36:51.674952 (+ 1266us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:51.674965 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:51.674988 (+ 23us) server_negotiation.cc:407] Connection header received
0806 11:36:51.675103 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:51.675109 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:51.675347 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:51.676086 (+ 739us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:36:51.676244 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:51.677364 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:51.679071 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:51.679954 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:51.680074 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:51.680240 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:51.680353 (+ 113us) server_negotiation.cc:299] Negotiation successful
0806 11:36:51.680473 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1076,"thread_start_us":111,"threads_started":1}
I20260806 11:36:51.681476 30099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:51.674398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46500 (local address 127.21.14.195:38073)
0806 11:36:51.675035 (+ 637us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:51.675050 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:51.675582 (+ 532us) server_negotiation.cc:407] Connection header received
0806 11:36:51.675669 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:51.675675 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:51.675906 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:51.676104 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:36:51.676346 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:51.677095 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:51.679071 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:51.679954 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:51.680074 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:51.680240 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:51.680944 (+ 704us) server_negotiation.cc:299] Negotiation successful
0806 11:36:51.681319 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":518,"thread_start_us":118,"threads_started":1}
I20260806 11:36:51.684561 29837 tablet_service.cc:1467] Processing CreateTablet for tablet 000360b2f1594347a6cf777b03387739 (DEFAULT_TABLE table=test-table [id=b114aeb43b0b486db8ee5d8213cd5870]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:36:51.684563 29995 tablet_service.cc:1467] Processing CreateTablet for tablet 000360b2f1594347a6cf777b03387739 (DEFAULT_TABLE table=test-table [id=b114aeb43b0b486db8ee5d8213cd5870]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:36:51.684561 29690 tablet_service.cc:1467] Processing CreateTablet for tablet 000360b2f1594347a6cf777b03387739 (DEFAULT_TABLE table=test-table [id=b114aeb43b0b486db8ee5d8213cd5870]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:36:51.685779 29837 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 000360b2f1594347a6cf777b03387739. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:51.685791 29690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 000360b2f1594347a6cf777b03387739. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:51.685791 29995 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 000360b2f1594347a6cf777b03387739. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:51.694783 30102 tablet_bootstrap.cc:492] T 000360b2f1594347a6cf777b03387739 P f1d856374ce44857a183737b5e2b03ba: Bootstrap starting.
I20260806 11:36:51.695089 30101 tablet_bootstrap.cc:492] T 000360b2f1594347a6cf777b03387739 P 741edc4087df42139936dace73d5f68a: Bootstrap starting.
I20260806 11:36:51.696125 30100 tablet_bootstrap.cc:492] T 000360b2f1594347a6cf777b03387739 P 92712864d28043d5aa2771fbc08fc004: Bootstrap starting.
I20260806 11:36:51.697860 30101 tablet_bootstrap.cc:654] T 000360b2f1594347a6cf777b03387739 P 741edc4087df42139936dace73d5f68a: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:51.697860 30102 tablet_bootstrap.cc:654] T 000360b2f1594347a6cf777b03387739 P f1d856374ce44857a183737b5e2b03ba: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:51.698280 30100 tablet_bootstrap.cc:654] T 000360b2f1594347a6cf777b03387739 P 92712864d28043d5aa2771fbc08fc004: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:51.698920 30102 log.cc:824] T 000360b2f1594347a6cf777b03387739 P f1d856374ce44857a183737b5e2b03ba: Log is configured to *not* fsync() on all Append() calls
I20260806 11:36:51.698920 30101 log.cc:824] T 000360b2f1594347a6cf777b03387739 P 741edc4087df42139936dace73d5f68a: Log is configured to *not* fsync() on all Append() calls
I20260806 11:36:51.699218 30100 log.cc:824] T 000360b2f1594347a6cf777b03387739 P 92712864d28043d5aa2771fbc08fc004: Log is configured to *not* fsync() on all Append() calls
I20260806 11:36:51.701331 30100 tablet_bootstrap.cc:492] T 000360b2f1594347a6cf777b03387739 P 92712864d28043d5aa2771fbc08fc004: No bootstrap required, opened a new log
I20260806 11:36:51.701440 30101 tablet_bootstrap.cc:492] T 000360b2f1594347a6cf777b03387739 P 741edc4087df42139936dace73d5f68a: No bootstrap required, opened a new log
I20260806 11:36:51.701440 30102 tablet_bootstrap.cc:492] T 000360b2f1594347a6cf777b03387739 P f1d856374ce44857a183737b5e2b03ba: No bootstrap required, opened a new log
I20260806 11:36:51.701529 30100 ts_tablet_manager.cc:1397] T 000360b2f1594347a6cf777b03387739 P 92712864d28043d5aa2771fbc08fc004: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 11:36:51.701648 30102 ts_tablet_manager.cc:1397] T 000360b2f1594347a6cf777b03387739 P f1d856374ce44857a183737b5e2b03ba: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260806 11:36:51.701648 30101 ts_tablet_manager.cc:1397] T 000360b2f1594347a6cf777b03387739 P 741edc4087df42139936dace73d5f68a: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 11:36:51.705045 30100 raft_consensus.cc:348] T 000360b2f1594347a6cf777b03387739 P 92712864d28043d5aa2771fbc08fc004 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92712864d28043d5aa2771fbc08fc004" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37297 } } peers { permanent_uuid: "741edc4087df42139936dace73d5f68a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40189 } } peers { permanent_uuid: "f1d856374ce44857a183737b5e2b03ba" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38073 } }
I20260806 11:36:51.705420 30100 raft_consensus.cc:374] T 000360b2f1594347a6cf777b03387739 P 92712864d28043d5aa2771fbc08fc004 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:51.705492 30100 raft_consensus.cc:729] T 000360b2f1594347a6cf777b03387739 P 92712864d28043d5aa2771fbc08fc004 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92712864d28043d5aa2771fbc08fc004, State: Initialized, Role: FOLLOWER
I20260806 11:36:51.706030 30100 consensus_queue.cc:260] T 000360b2f1594347a6cf777b03387739 P 92712864d28043d5aa2771fbc08fc004 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92712864d28043d5aa2771fbc08fc004" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37297 } } peers { permanent_uuid: "741edc4087df42139936dace73d5f68a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40189 } } peers { permanent_uuid: "f1d856374ce44857a183737b5e2b03ba" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38073 } }
I20260806 11:36:51.706270 30102 raft_consensus.cc:348] T 000360b2f1594347a6cf777b03387739 P f1d856374ce44857a183737b5e2b03ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92712864d28043d5aa2771fbc08fc004" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37297 } } peers { permanent_uuid: "741edc4087df42139936dace73d5f68a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40189 } } peers { permanent_uuid: "f1d856374ce44857a183737b5e2b03ba" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38073 } }
I20260806 11:36:51.706648 30102 raft_consensus.cc:374] T 000360b2f1594347a6cf777b03387739 P f1d856374ce44857a183737b5e2b03ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:51.706710 30102 raft_consensus.cc:729] T 000360b2f1594347a6cf777b03387739 P f1d856374ce44857a183737b5e2b03ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1d856374ce44857a183737b5e2b03ba, State: Initialized, Role: FOLLOWER
I20260806 11:36:51.706812 30101 raft_consensus.cc:348] T 000360b2f1594347a6cf777b03387739 P 741edc4087df42139936dace73d5f68a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92712864d28043d5aa2771fbc08fc004" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37297 } } peers { permanent_uuid: "741edc4087df42139936dace73d5f68a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40189 } } peers { permanent_uuid: "f1d856374ce44857a183737b5e2b03ba" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38073 } }
I20260806 11:36:51.707108 30101 raft_consensus.cc:374] T 000360b2f1594347a6cf777b03387739 P 741edc4087df42139936dace73d5f68a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:51.707159 30101 raft_consensus.cc:729] T 000360b2f1594347a6cf777b03387739 P 741edc4087df42139936dace73d5f68a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 741edc4087df42139936dace73d5f68a, State: Initialized, Role: FOLLOWER
I20260806 11:36:51.707203 29910 heartbeater.cc:503] Master 127.21.14.254:35125 was elected leader, sending a full tablet report...
I20260806 11:36:51.707268 30100 ts_tablet_manager.cc:1428] T 000360b2f1594347a6cf777b03387739 P 92712864d28043d5aa2771fbc08fc004: Time spent starting tablet: real 0.006s user 0.002s sys 0.005s
I20260806 11:36:51.707245 30102 consensus_queue.cc:260] T 000360b2f1594347a6cf777b03387739 P f1d856374ce44857a183737b5e2b03ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92712864d28043d5aa2771fbc08fc004" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37297 } } peers { permanent_uuid: "741edc4087df42139936dace73d5f68a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40189 } } peers { permanent_uuid: "f1d856374ce44857a183737b5e2b03ba" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38073 } }
I20260806 11:36:51.707664 30101 consensus_queue.cc:260] T 000360b2f1594347a6cf777b03387739 P 741edc4087df42139936dace73d5f68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92712864d28043d5aa2771fbc08fc004" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37297 } } peers { permanent_uuid: "741edc4087df42139936dace73d5f68a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40189 } } peers { permanent_uuid: "f1d856374ce44857a183737b5e2b03ba" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38073 } }
I20260806 11:36:51.708133 30069 heartbeater.cc:503] Master 127.21.14.254:35125 was elected leader, sending a full tablet report...
I20260806 11:36:51.708556 30102 ts_tablet_manager.cc:1428] T 000360b2f1594347a6cf777b03387739 P f1d856374ce44857a183737b5e2b03ba: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260806 11:36:51.708555 30101 ts_tablet_manager.cc:1428] T 000360b2f1594347a6cf777b03387739 P 741edc4087df42139936dace73d5f68a: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260806 11:36:51.708909 29759 heartbeater.cc:503] Master 127.21.14.254:35125 was elected leader, sending a full tablet report...
I20260806 11:36:51.756285 30108 raft_consensus.cc:482] T 000360b2f1594347a6cf777b03387739 P 741edc4087df42139936dace73d5f68a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 11:36:51.756533 30108 raft_consensus.cc:504] T 000360b2f1594347a6cf777b03387739 P 741edc4087df42139936dace73d5f68a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92712864d28043d5aa2771fbc08fc004" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37297 } } peers { permanent_uuid: "741edc4087df42139936dace73d5f68a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40189 } } peers { permanent_uuid: "f1d856374ce44857a183737b5e2b03ba" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38073 } }
I20260806 11:36:51.758050 30108 leader_election.cc:288] T 000360b2f1594347a6cf777b03387739 P 741edc4087df42139936dace73d5f68a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 92712864d28043d5aa2771fbc08fc004 (127.21.14.194:37297), f1d856374ce44857a183737b5e2b03ba (127.21.14.195:38073)
I20260806 11:36:51.763636 30095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:51.758446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:52299 (local address 127.21.14.194:37297)
0806 11:36:51.758602 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:51.758609 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:51.758914 (+ 305us) server_negotiation.cc:407] Connection header received
0806 11:36:51.759103 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:51.759106 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:51.759151 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:51.759223 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:36:51.759531 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:51.760366 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:51.762265 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:51.762925 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:51.763060 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:51.763118 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:51.763408 (+ 290us) server_negotiation.cc:299] Negotiation successful
0806 11:36:51.763487 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 11:36:51.763636 30099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:51.758795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:56177 (local address 127.21.14.195:38073)
0806 11:36:51.758946 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:51.758950 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:51.758965 (+ 15us) server_negotiation.cc:407] Connection header received
0806 11:36:51.759339 (+ 374us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:51.759342 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:51.759380 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:51.759436 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:36:51.759729 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:51.760458 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:51.762265 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:51.762925 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:51.763061 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:51.763118 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:51.763408 (+ 290us) server_negotiation.cc:299] Negotiation successful
0806 11:36:51.763487 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 11:36:51.763705 30110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:51.758298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.194:37297 (local address 127.21.14.193:52299)
0806 11:36:51.758774 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:51.758789 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:51.758971 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:51.759253 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:51.759256 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:51.759272 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:36:51.759382 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:51.759387 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:51.760494 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:51.760497 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:51.762119 (+ 1622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:51.762131 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:51.763071 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:51.763075 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:51.763240 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:51.763287 (+ 47us) client_negotiation.cc:769] Sending connection context
0806 11:36:51.763374 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 11:36:51.763498 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":403,"mutex_wait_us":224,"thread_start_us":144,"threads_started":1}
I20260806 11:36:51.763711 30109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:51.758350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.195:38073 (local address 127.21.14.193:56177)
0806 11:36:51.758801 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:51.758816 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:51.758969 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:51.759476 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:51.759482 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:51.759502 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:36:51.759610 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:51.759616 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:51.760569 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:51.760574 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:51.762117 (+ 1543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:51.762131 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:51.763071 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:51.763075 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:51.763236 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:51.763295 (+ 59us) client_negotiation.cc:769] Sending connection context
0806 11:36:51.763374 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 11:36:51.763504 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":399,"mutex_wait_us":163,"thread_start_us":119,"threads_started":1}
I20260806 11:36:51.764518 29859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "000360b2f1594347a6cf777b03387739" candidate_uuid: "741edc4087df42139936dace73d5f68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92712864d28043d5aa2771fbc08fc004" is_pre_election: true
I20260806 11:36:51.764613 30018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "000360b2f1594347a6cf777b03387739" candidate_uuid: "741edc4087df42139936dace73d5f68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1d856374ce44857a183737b5e2b03ba" is_pre_election: true
I20260806 11:36:51.764899 29859 raft_consensus.cc:2454] T 000360b2f1594347a6cf777b03387739 P 92712864d28043d5aa2771fbc08fc004 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 741edc4087df42139936dace73d5f68a in term 0.
I20260806 11:36:51.764920 30018 raft_consensus.cc:2454] T 000360b2f1594347a6cf777b03387739 P f1d856374ce44857a183737b5e2b03ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 741edc4087df42139936dace73d5f68a in term 0.
I20260806 11:36:51.765511 29638 leader_election.cc:302] T 000360b2f1594347a6cf777b03387739 P 741edc4087df42139936dace73d5f68a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 741edc4087df42139936dace73d5f68a, 92712864d28043d5aa2771fbc08fc004, f1d856374ce44857a183737b5e2b03ba; no voters:
I20260806 11:36:51.765875 30108 raft_consensus.cc:2781] T 000360b2f1594347a6cf777b03387739 P 741edc4087df42139936dace73d5f68a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 11:36:51.765975 30108 raft_consensus.cc:482] T 000360b2f1594347a6cf777b03387739 P 741edc4087df42139936dace73d5f68a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 11:36:51.766021 30108 raft_consensus.cc:3037] T 000360b2f1594347a6cf777b03387739 P 741edc4087df42139936dace73d5f68a [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:36:51.767194 30108 raft_consensus.cc:504] T 000360b2f1594347a6cf777b03387739 P 741edc4087df42139936dace73d5f68a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92712864d28043d5aa2771fbc08fc004" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37297 } } peers { permanent_uuid: "741edc4087df42139936dace73d5f68a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40189 } } peers { permanent_uuid: "f1d856374ce44857a183737b5e2b03ba" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38073 } }
I20260806 11:36:51.767628 30108 leader_election.cc:288] T 000360b2f1594347a6cf777b03387739 P 741edc4087df42139936dace73d5f68a [CANDIDATE]: Term 1 election: Requested vote from peers 92712864d28043d5aa2771fbc08fc004 (127.21.14.194:37297), f1d856374ce44857a183737b5e2b03ba (127.21.14.195:38073)
I20260806 11:36:51.768071 29859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "000360b2f1594347a6cf777b03387739" candidate_uuid: "741edc4087df42139936dace73d5f68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92712864d28043d5aa2771fbc08fc004"
I20260806 11:36:51.768071 30018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "000360b2f1594347a6cf777b03387739" candidate_uuid: "741edc4087df42139936dace73d5f68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1d856374ce44857a183737b5e2b03ba"
I20260806 11:36:51.768224 30018 raft_consensus.cc:3037] T 000360b2f1594347a6cf777b03387739 P f1d856374ce44857a183737b5e2b03ba [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:36:51.768224 29859 raft_consensus.cc:3037] T 000360b2f1594347a6cf777b03387739 P 92712864d28043d5aa2771fbc08fc004 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:36:51.769794 30018 raft_consensus.cc:2454] T 000360b2f1594347a6cf777b03387739 P f1d856374ce44857a183737b5e2b03ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 741edc4087df42139936dace73d5f68a in term 1.
I20260806 11:36:51.770219 29859 raft_consensus.cc:2454] T 000360b2f1594347a6cf777b03387739 P 92712864d28043d5aa2771fbc08fc004 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 741edc4087df42139936dace73d5f68a in term 1.
I20260806 11:36:51.770287 29640 leader_election.cc:302] T 000360b2f1594347a6cf777b03387739 P 741edc4087df42139936dace73d5f68a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 741edc4087df42139936dace73d5f68a, f1d856374ce44857a183737b5e2b03ba; no voters:
I20260806 11:36:51.770520 30108 raft_consensus.cc:2781] T 000360b2f1594347a6cf777b03387739 P 741edc4087df42139936dace73d5f68a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:36:51.770783 30108 raft_consensus.cc:686] T 000360b2f1594347a6cf777b03387739 P 741edc4087df42139936dace73d5f68a [term 1 LEADER]: Becoming Leader. State: Replica: 741edc4087df42139936dace73d5f68a, State: Running, Role: LEADER
I20260806 11:36:51.771240 30108 consensus_queue.cc:237] T 000360b2f1594347a6cf777b03387739 P 741edc4087df42139936dace73d5f68a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92712864d28043d5aa2771fbc08fc004" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37297 } } peers { permanent_uuid: "741edc4087df42139936dace73d5f68a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40189 } } peers { permanent_uuid: "f1d856374ce44857a183737b5e2b03ba" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38073 } }
I20260806 11:36:51.775403 29543 catalog_manager.cc:5495] T 000360b2f1594347a6cf777b03387739 P 741edc4087df42139936dace73d5f68a reported cstate change: term changed from 0 to 1, leader changed from <none> to 741edc4087df42139936dace73d5f68a (127.21.14.193). New cstate: current_term: 1 leader_uuid: "741edc4087df42139936dace73d5f68a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92712864d28043d5aa2771fbc08fc004" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37297 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "741edc4087df42139936dace73d5f68a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f1d856374ce44857a183737b5e2b03ba" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38073 } health_report { overall_health: UNKNOWN } } }
I20260806 11:36:51.809880 30098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:51.805240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34192 (local address 127.21.14.193:40189)
0806 11:36:51.805406 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:51.805411 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:51.805426 (+ 15us) server_negotiation.cc:407] Connection header received
0806 11:36:51.805668 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:51.805671 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:51.805730 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:51.805873 (+ 143us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 11:36:51.806170 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:51.806708 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:51.807889 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:51.808155 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:51.808252 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:51.808704 (+ 452us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 11:36:51.809307 (+ 603us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 11:36:51.809460 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:51.809609 (+ 149us) server_negotiation.cc:299] Negotiation successful
0806 11:36:51.809680 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 11:36:51.817248 29859 raft_consensus.cc:1261] T 000360b2f1594347a6cf777b03387739 P 92712864d28043d5aa2771fbc08fc004 [term 1 FOLLOWER]: Refusing update from remote peer 741edc4087df42139936dace73d5f68a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 11:36:51.817248 30018 raft_consensus.cc:1261] T 000360b2f1594347a6cf777b03387739 P f1d856374ce44857a183737b5e2b03ba [term 1 FOLLOWER]: Refusing update from remote peer 741edc4087df42139936dace73d5f68a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 11:36:51.818398 30108 consensus_queue.cc:1035] T 000360b2f1594347a6cf777b03387739 P 741edc4087df42139936dace73d5f68a [LEADER]: Connected to new peer: Peer: permanent_uuid: "92712864d28043d5aa2771fbc08fc004" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 11:36:51.818697 30111 consensus_queue.cc:1035] T 000360b2f1594347a6cf777b03387739 P 741edc4087df42139936dace73d5f68a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1d856374ce44857a183737b5e2b03ba" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38073 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 11:36:51.830933 30116 mvcc.cc:205] Tried to move back new op lower bound from 7315522403593412608 to 7315522403418193920. Current Snapshot: MvccSnapshot[applied={T|T < 7315522403593412608}]
I20260806 11:36:51.831218 30115 mvcc.cc:205] Tried to move back new op lower bound from 7315522403593412608 to 7315522403418193920. Current Snapshot: MvccSnapshot[applied={T|T < 7315522403593412608}]
I20260806 11:36:51.841898 29541 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60574:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 11:36:51.842111 29541 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60574:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 11:36:51.845546 29541 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ed204855a8dd4cd686fd550178b9563e: Sending DeleteTablet for 3 replicas of tablet 000360b2f1594347a6cf777b03387739
I20260806 11:36:51.846679 29837 tablet_service.cc:1514] Processing DeleteTablet for tablet 000360b2f1594347a6cf777b03387739 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:36:51 UTC) from {username='oryx', principal='oryx/127.21.14.254@KRBTEST.COM'} at 127.0.0.1:60302
I20260806 11:36:51.846701 29690 tablet_service.cc:1514] Processing DeleteTablet for tablet 000360b2f1594347a6cf777b03387739 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:36:51 UTC) from {username='oryx', principal='oryx/127.21.14.254@KRBTEST.COM'} at 127.0.0.1:34180
I20260806 11:36:51.846884 29995 tablet_service.cc:1514] Processing DeleteTablet for tablet 000360b2f1594347a6cf777b03387739 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:36:51 UTC) from {username='oryx', principal='oryx/127.21.14.254@KRBTEST.COM'} at 127.0.0.1:46500
I20260806 11:36:51.847316 30124 tablet_replica.cc:330] T 000360b2f1594347a6cf777b03387739 P 92712864d28043d5aa2771fbc08fc004: stopping tablet replica
I20260806 11:36:51.847622 30125 tablet_replica.cc:330] T 000360b2f1594347a6cf777b03387739 P 741edc4087df42139936dace73d5f68a: stopping tablet replica
I20260806 11:36:51.847755 30124 raft_consensus.cc:2229] T 000360b2f1594347a6cf777b03387739 P 92712864d28043d5aa2771fbc08fc004 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 11:36:51.847930 30125 raft_consensus.cc:2229] T 000360b2f1594347a6cf777b03387739 P 741edc4087df42139936dace73d5f68a [term 1 LEADER]: Raft consensus shutting down.
I20260806 11:36:51.848129 30124 raft_consensus.cc:2258] T 000360b2f1594347a6cf777b03387739 P 92712864d28043d5aa2771fbc08fc004 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 11:36:51.848383 30125 raft_consensus.cc:2258] T 000360b2f1594347a6cf777b03387739 P 741edc4087df42139936dace73d5f68a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 11:36:51.848335 29541 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60574:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 11:36:51.848767 30126 tablet_replica.cc:330] T 000360b2f1594347a6cf777b03387739 P f1d856374ce44857a183737b5e2b03ba: stopping tablet replica
W20260806 11:36:51.848867 29541 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 11:36:51.849153 30126 raft_consensus.cc:2229] T 000360b2f1594347a6cf777b03387739 P f1d856374ce44857a183737b5e2b03ba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 11:36:51.849491 30126 raft_consensus.cc:2258] T 000360b2f1594347a6cf777b03387739 P f1d856374ce44857a183737b5e2b03ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 11:36:51.850045 30124 ts_tablet_manager.cc:1905] T 000360b2f1594347a6cf777b03387739 P 92712864d28043d5aa2771fbc08fc004: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 11:36:51.852255 30125 ts_tablet_manager.cc:1905] T 000360b2f1594347a6cf777b03387739 P 741edc4087df42139936dace73d5f68a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 11:36:51.853737 30124 ts_tablet_manager.cc:1918] T 000360b2f1594347a6cf777b03387739 P 92712864d28043d5aa2771fbc08fc004: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 11:36:51.853884 30124 log.cc:1196] T 000360b2f1594347a6cf777b03387739 P 92712864d28043d5aa2771fbc08fc004: Deleting WAL directory at /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/ts-1/wal/wals/000360b2f1594347a6cf777b03387739
I20260806 11:36:51.854282 30124 ts_tablet_manager.cc:1939] T 000360b2f1594347a6cf777b03387739 P 92712864d28043d5aa2771fbc08fc004: Deleting consensus metadata
I20260806 11:36:51.856042 29837 tablet_service.cc:1467] Processing CreateTablet for tablet fe2f39af2fd1453597dab9175511b94c (DEFAULT_TABLE table=test-table [id=4d97ed31ab1349eab491472ed191a36f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:36:51.856043 29995 tablet_service.cc:1467] Processing CreateTablet for tablet fe2f39af2fd1453597dab9175511b94c (DEFAULT_TABLE table=test-table [id=4d97ed31ab1349eab491472ed191a36f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:36:51.856110 29690 tablet_service.cc:1467] Processing CreateTablet for tablet fe2f39af2fd1453597dab9175511b94c (DEFAULT_TABLE table=test-table [id=4d97ed31ab1349eab491472ed191a36f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:36:51.856382 29837 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe2f39af2fd1453597dab9175511b94c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:51.856381 29995 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe2f39af2fd1453597dab9175511b94c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:51.856412 29690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe2f39af2fd1453597dab9175511b94c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:51.859012 30101 tablet_bootstrap.cc:492] T fe2f39af2fd1453597dab9175511b94c P 741edc4087df42139936dace73d5f68a: Bootstrap starting.
I20260806 11:36:51.859061 30100 tablet_bootstrap.cc:492] T fe2f39af2fd1453597dab9175511b94c P 92712864d28043d5aa2771fbc08fc004: Bootstrap starting.
I20260806 11:36:51.860256 30100 tablet_bootstrap.cc:654] T fe2f39af2fd1453597dab9175511b94c P 92712864d28043d5aa2771fbc08fc004: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:51.860389 30101 tablet_bootstrap.cc:654] T fe2f39af2fd1453597dab9175511b94c P 741edc4087df42139936dace73d5f68a: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:51.860332 29528 catalog_manager.cc:4841] TS 92712864d28043d5aa2771fbc08fc004 (127.21.14.194:37297): tablet 000360b2f1594347a6cf777b03387739 (table test-table [id=b114aeb43b0b486db8ee5d8213cd5870]) successfully deleted
I20260806 11:36:51.862195 30102 tablet_bootstrap.cc:492] T fe2f39af2fd1453597dab9175511b94c P f1d856374ce44857a183737b5e2b03ba: Bootstrap starting.
I20260806 11:36:51.862552 30101 tablet_bootstrap.cc:492] T fe2f39af2fd1453597dab9175511b94c P 741edc4087df42139936dace73d5f68a: No bootstrap required, opened a new log
I20260806 11:36:51.862648 30101 ts_tablet_manager.cc:1397] T fe2f39af2fd1453597dab9175511b94c P 741edc4087df42139936dace73d5f68a: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260806 11:36:51.862721 30126 ts_tablet_manager.cc:1905] T 000360b2f1594347a6cf777b03387739 P f1d856374ce44857a183737b5e2b03ba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 11:36:51.863200 30101 raft_consensus.cc:348] T fe2f39af2fd1453597dab9175511b94c P 741edc4087df42139936dace73d5f68a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741edc4087df42139936dace73d5f68a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40189 } } peers { permanent_uuid: "92712864d28043d5aa2771fbc08fc004" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37297 } } peers { permanent_uuid: "f1d856374ce44857a183737b5e2b03ba" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38073 } }
I20260806 11:36:51.863325 30101 raft_consensus.cc:374] T fe2f39af2fd1453597dab9175511b94c P 741edc4087df42139936dace73d5f68a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:51.863323 30102 tablet_bootstrap.cc:654] T fe2f39af2fd1453597dab9175511b94c P f1d856374ce44857a183737b5e2b03ba: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:51.863356 30101 raft_consensus.cc:729] T fe2f39af2fd1453597dab9175511b94c P 741edc4087df42139936dace73d5f68a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 741edc4087df42139936dace73d5f68a, State: Initialized, Role: FOLLOWER
I20260806 11:36:51.863724 30101 consensus_queue.cc:260] T fe2f39af2fd1453597dab9175511b94c P 741edc4087df42139936dace73d5f68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741edc4087df42139936dace73d5f68a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40189 } } peers { permanent_uuid: "92712864d28043d5aa2771fbc08fc004" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37297 } } peers { permanent_uuid: "f1d856374ce44857a183737b5e2b03ba" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38073 } }
I20260806 11:36:51.865479 30102 tablet_bootstrap.cc:492] T fe2f39af2fd1453597dab9175511b94c P f1d856374ce44857a183737b5e2b03ba: No bootstrap required, opened a new log
I20260806 11:36:51.865592 30102 ts_tablet_manager.cc:1397] T fe2f39af2fd1453597dab9175511b94c P f1d856374ce44857a183737b5e2b03ba: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260806 11:36:51.866021 30126 ts_tablet_manager.cc:1918] T 000360b2f1594347a6cf777b03387739 P f1d856374ce44857a183737b5e2b03ba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 11:36:51.866135 30126 log.cc:1196] T 000360b2f1594347a6cf777b03387739 P f1d856374ce44857a183737b5e2b03ba: Deleting WAL directory at /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/ts-2/wal/wals/000360b2f1594347a6cf777b03387739
I20260806 11:36:51.866195 30102 raft_consensus.cc:348] T fe2f39af2fd1453597dab9175511b94c P f1d856374ce44857a183737b5e2b03ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741edc4087df42139936dace73d5f68a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40189 } } peers { permanent_uuid: "92712864d28043d5aa2771fbc08fc004" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37297 } } peers { permanent_uuid: "f1d856374ce44857a183737b5e2b03ba" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38073 } }
I20260806 11:36:51.866340 30102 raft_consensus.cc:374] T fe2f39af2fd1453597dab9175511b94c P f1d856374ce44857a183737b5e2b03ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:51.866387 30102 raft_consensus.cc:729] T fe2f39af2fd1453597dab9175511b94c P f1d856374ce44857a183737b5e2b03ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1d856374ce44857a183737b5e2b03ba, State: Initialized, Role: FOLLOWER
I20260806 11:36:51.866518 30126 ts_tablet_manager.cc:1939] T 000360b2f1594347a6cf777b03387739 P f1d856374ce44857a183737b5e2b03ba: Deleting consensus metadata
I20260806 11:36:51.866549 30102 consensus_queue.cc:260] T fe2f39af2fd1453597dab9175511b94c P f1d856374ce44857a183737b5e2b03ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741edc4087df42139936dace73d5f68a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40189 } } peers { permanent_uuid: "92712864d28043d5aa2771fbc08fc004" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37297 } } peers { permanent_uuid: "f1d856374ce44857a183737b5e2b03ba" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38073 } }
I20260806 11:36:51.866541 30101 ts_tablet_manager.cc:1428] T fe2f39af2fd1453597dab9175511b94c P 741edc4087df42139936dace73d5f68a: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20260806 11:36:51.866832 30102 ts_tablet_manager.cc:1428] T fe2f39af2fd1453597dab9175511b94c P f1d856374ce44857a183737b5e2b03ba: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 11:36:51.867933 29530 catalog_manager.cc:4841] TS f1d856374ce44857a183737b5e2b03ba (127.21.14.195:38073): tablet 000360b2f1594347a6cf777b03387739 (table test-table [id=b114aeb43b0b486db8ee5d8213cd5870]) successfully deleted
I20260806 11:36:51.868537 30125 ts_tablet_manager.cc:1918] T 000360b2f1594347a6cf777b03387739 P 741edc4087df42139936dace73d5f68a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 11:36:51.868968 30125 log.cc:1196] T 000360b2f1594347a6cf777b03387739 P 741edc4087df42139936dace73d5f68a: Deleting WAL directory at /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/ts-0/wal/wals/000360b2f1594347a6cf777b03387739
I20260806 11:36:51.869370 30125 ts_tablet_manager.cc:1939] T 000360b2f1594347a6cf777b03387739 P 741edc4087df42139936dace73d5f68a: Deleting consensus metadata
I20260806 11:36:51.870635 29527 catalog_manager.cc:4841] TS 741edc4087df42139936dace73d5f68a (127.21.14.193:40189): tablet 000360b2f1594347a6cf777b03387739 (table test-table [id=b114aeb43b0b486db8ee5d8213cd5870]) successfully deleted
I20260806 11:36:51.871795 30100 tablet_bootstrap.cc:492] T fe2f39af2fd1453597dab9175511b94c P 92712864d28043d5aa2771fbc08fc004: No bootstrap required, opened a new log
I20260806 11:36:51.871912 30100 ts_tablet_manager.cc:1397] T fe2f39af2fd1453597dab9175511b94c P 92712864d28043d5aa2771fbc08fc004: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.000s
I20260806 11:36:51.872452 30100 raft_consensus.cc:348] T fe2f39af2fd1453597dab9175511b94c P 92712864d28043d5aa2771fbc08fc004 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741edc4087df42139936dace73d5f68a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40189 } } peers { permanent_uuid: "92712864d28043d5aa2771fbc08fc004" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37297 } } peers { permanent_uuid: "f1d856374ce44857a183737b5e2b03ba" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38073 } }
I20260806 11:36:51.872581 30100 raft_consensus.cc:374] T fe2f39af2fd1453597dab9175511b94c P 92712864d28043d5aa2771fbc08fc004 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:51.872609 30100 raft_consensus.cc:729] T fe2f39af2fd1453597dab9175511b94c P 92712864d28043d5aa2771fbc08fc004 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92712864d28043d5aa2771fbc08fc004, State: Initialized, Role: FOLLOWER
I20260806 11:36:51.872745 30100 consensus_queue.cc:260] T fe2f39af2fd1453597dab9175511b94c P 92712864d28043d5aa2771fbc08fc004 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741edc4087df42139936dace73d5f68a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40189 } } peers { permanent_uuid: "92712864d28043d5aa2771fbc08fc004" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37297 } } peers { permanent_uuid: "f1d856374ce44857a183737b5e2b03ba" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38073 } }
I20260806 11:36:51.873023 30100 ts_tablet_manager.cc:1428] T fe2f39af2fd1453597dab9175511b94c P 92712864d28043d5aa2771fbc08fc004: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 11:36:51.982641 30106 raft_consensus.cc:482] T fe2f39af2fd1453597dab9175511b94c P 92712864d28043d5aa2771fbc08fc004 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 11:36:51.982832 30106 raft_consensus.cc:504] T fe2f39af2fd1453597dab9175511b94c P 92712864d28043d5aa2771fbc08fc004 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741edc4087df42139936dace73d5f68a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40189 } } peers { permanent_uuid: "92712864d28043d5aa2771fbc08fc004" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37297 } } peers { permanent_uuid: "f1d856374ce44857a183737b5e2b03ba" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38073 } }
I20260806 11:36:51.984270 30106 leader_election.cc:288] T fe2f39af2fd1453597dab9175511b94c P 92712864d28043d5aa2771fbc08fc004 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 741edc4087df42139936dace73d5f68a (127.21.14.193:40189), f1d856374ce44857a183737b5e2b03ba (127.21.14.195:38073)
I20260806 11:36:51.990091 30098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:51.984751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:51683 (local address 127.21.14.193:40189)
0806 11:36:51.984893 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:51.984899 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:51.984993 (+ 94us) server_negotiation.cc:407] Connection header received
0806 11:36:51.985237 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:51.985243 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:51.985304 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:51.985361 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:36:51.985898 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:51.986691 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:51.988338 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:51.988991 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:51.989138 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:51.989480 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:51.989766 (+ 286us) server_negotiation.cc:299] Negotiation successful
0806 11:36:51.989910 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 11:36:51.990679 30130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:51.984528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.193:40189 (local address 127.21.14.194:51683)
0806 11:36:51.984890 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:51.984909 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:51.985090 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:51.985559 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:51.985562 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:51.985582 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:36:51.985769 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:51.985776 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:51.986963 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:51.986967 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:51.988177 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:51.988192 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:51.989349 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:51.989364 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:51.989490 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:51.989539 (+ 49us) client_negotiation.cc:769] Sending connection context
0806 11:36:51.990370 (+ 831us) client_negotiation.cc:240] Negotiation successful
0806 11:36:51.990479 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"mutex_wait_us":24,"spinlock_wait_cycles":7936,"thread_start_us":86,"threads_started":1}
I20260806 11:36:51.991358 30099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:51.984991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:43935 (local address 127.21.14.195:38073)
0806 11:36:51.985251 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:51.985256 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:51.985276 (+ 20us) server_negotiation.cc:407] Connection header received
0806 11:36:51.985397 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:51.985400 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:51.985450 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:51.985572 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:36:51.986914 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:51.987666 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:51.989417 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:51.990022 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:51.990156 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:51.990222 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:51.991107 (+ 885us) server_negotiation.cc:299] Negotiation successful
0806 11:36:51.991218 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175}
I20260806 11:36:51.991540 29710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe2f39af2fd1453597dab9175511b94c" candidate_uuid: "92712864d28043d5aa2771fbc08fc004" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "741edc4087df42139936dace73d5f68a" is_pre_election: true
I20260806 11:36:51.991942 30131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:51.984602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.195:38073 (local address 127.21.14.194:43935)
0806 11:36:51.985201 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:51.985213 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:51.985284 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:51.985572 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:51.985574 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:51.985596 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:36:51.985755 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:51.985766 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:51.987820 (+ 2054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:51.987825 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:51.988993 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:51.989039 (+ 46us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:51.990555 (+ 1516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:51.990559 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:51.990658 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:51.990687 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 11:36:51.991076 (+ 389us) client_negotiation.cc:240] Negotiation successful
0806 11:36:51.991319 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"mutex_wait_us":24,"spinlock_wait_cycles":768,"thread_start_us":101,"threads_started":1}
I20260806 11:36:51.992033 29710 raft_consensus.cc:2454] T fe2f39af2fd1453597dab9175511b94c P 741edc4087df42139936dace73d5f68a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92712864d28043d5aa2771fbc08fc004 in term 0.
I20260806 11:36:51.992568 29785 leader_election.cc:302] T fe2f39af2fd1453597dab9175511b94c P 92712864d28043d5aa2771fbc08fc004 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 741edc4087df42139936dace73d5f68a, 92712864d28043d5aa2771fbc08fc004; no voters:
I20260806 11:36:51.992633 30018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe2f39af2fd1453597dab9175511b94c" candidate_uuid: "92712864d28043d5aa2771fbc08fc004" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1d856374ce44857a183737b5e2b03ba" is_pre_election: true
I20260806 11:36:51.992802 30018 raft_consensus.cc:2454] T fe2f39af2fd1453597dab9175511b94c P f1d856374ce44857a183737b5e2b03ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92712864d28043d5aa2771fbc08fc004 in term 0.
I20260806 11:36:51.992930 30106 raft_consensus.cc:2781] T fe2f39af2fd1453597dab9175511b94c P 92712864d28043d5aa2771fbc08fc004 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 11:36:51.992990 30106 raft_consensus.cc:482] T fe2f39af2fd1453597dab9175511b94c P 92712864d28043d5aa2771fbc08fc004 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 11:36:51.993032 30106 raft_consensus.cc:3037] T fe2f39af2fd1453597dab9175511b94c P 92712864d28043d5aa2771fbc08fc004 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:36:51.994246 30106 raft_consensus.cc:504] T fe2f39af2fd1453597dab9175511b94c P 92712864d28043d5aa2771fbc08fc004 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741edc4087df42139936dace73d5f68a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40189 } } peers { permanent_uuid: "92712864d28043d5aa2771fbc08fc004" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37297 } } peers { permanent_uuid: "f1d856374ce44857a183737b5e2b03ba" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38073 } }
I20260806 11:36:51.994689 30106 leader_election.cc:288] T fe2f39af2fd1453597dab9175511b94c P 92712864d28043d5aa2771fbc08fc004 [CANDIDATE]: Term 1 election: Requested vote from peers 741edc4087df42139936dace73d5f68a (127.21.14.193:40189), f1d856374ce44857a183737b5e2b03ba (127.21.14.195:38073)
I20260806 11:36:51.995254 29710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe2f39af2fd1453597dab9175511b94c" candidate_uuid: "92712864d28043d5aa2771fbc08fc004" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "741edc4087df42139936dace73d5f68a"
I20260806 11:36:51.995283 30018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe2f39af2fd1453597dab9175511b94c" candidate_uuid: "92712864d28043d5aa2771fbc08fc004" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1d856374ce44857a183737b5e2b03ba"
I20260806 11:36:51.995429 30018 raft_consensus.cc:3037] T fe2f39af2fd1453597dab9175511b94c P f1d856374ce44857a183737b5e2b03ba [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:36:51.995424 29710 raft_consensus.cc:3037] T fe2f39af2fd1453597dab9175511b94c P 741edc4087df42139936dace73d5f68a [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:36:51.996459 29710 raft_consensus.cc:2454] T fe2f39af2fd1453597dab9175511b94c P 741edc4087df42139936dace73d5f68a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92712864d28043d5aa2771fbc08fc004 in term 1.
I20260806 11:36:51.996647 30018 raft_consensus.cc:2454] T fe2f39af2fd1453597dab9175511b94c P f1d856374ce44857a183737b5e2b03ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92712864d28043d5aa2771fbc08fc004 in term 1.
I20260806 11:36:51.996927 29785 leader_election.cc:302] T fe2f39af2fd1453597dab9175511b94c P 92712864d28043d5aa2771fbc08fc004 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 741edc4087df42139936dace73d5f68a, 92712864d28043d5aa2771fbc08fc004; no voters:
I20260806 11:36:51.997205 30106 raft_consensus.cc:2781] T fe2f39af2fd1453597dab9175511b94c P 92712864d28043d5aa2771fbc08fc004 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:36:51.997414 30106 raft_consensus.cc:686] T fe2f39af2fd1453597dab9175511b94c P 92712864d28043d5aa2771fbc08fc004 [term 1 LEADER]: Becoming Leader. State: Replica: 92712864d28043d5aa2771fbc08fc004, State: Running, Role: LEADER
I20260806 11:36:51.997699 30106 consensus_queue.cc:237] T fe2f39af2fd1453597dab9175511b94c P 92712864d28043d5aa2771fbc08fc004 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741edc4087df42139936dace73d5f68a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40189 } } peers { permanent_uuid: "92712864d28043d5aa2771fbc08fc004" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37297 } } peers { permanent_uuid: "f1d856374ce44857a183737b5e2b03ba" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38073 } }
I20260806 11:36:52.000391 29543 catalog_manager.cc:5495] T fe2f39af2fd1453597dab9175511b94c P 92712864d28043d5aa2771fbc08fc004 reported cstate change: term changed from 0 to 1, leader changed from <none> to 92712864d28043d5aa2771fbc08fc004 (127.21.14.194). New cstate: current_term: 1 leader_uuid: "92712864d28043d5aa2771fbc08fc004" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741edc4087df42139936dace73d5f68a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92712864d28043d5aa2771fbc08fc004" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37297 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f1d856374ce44857a183737b5e2b03ba" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38073 } health_report { overall_health: UNKNOWN } } }
Aug 06 11:36:52 dist-test-slave-7c35 krb5kdc[29497](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016210, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.14.254@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM
I20260806 11:36:52.057606 29596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:52.045125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60580 (local address 127.21.14.254:35125)
0806 11:36:52.045296 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:52.045303 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:52.045436 (+ 133us) server_negotiation.cc:407] Connection header received
0806 11:36:52.045720 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:52.045724 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:52.045787 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:52.045872 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:52.046740 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:52.047827 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:52.048846 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:52.049088 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:52.049288 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:52.053089 (+ 3801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:52.053093 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:52.053095 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:52.053126 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:52.055182 (+ 2056us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:52.055829 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:52.055832 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:52.055834 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:52.055920 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:52.056412 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:52.056415 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:52.056416 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:52.056636 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:52.056724 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:52.057295 (+ 571us) server_negotiation.cc:299] Negotiation successful
0806 11:36:52.057423 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260806 11:36:52.057610 30139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:52.044938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:35125 (local address 127.0.0.1:60580)
0806 11:36:52.045298 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:52.045317 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:52.045485 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:52.045908 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:52.045911 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:52.046476 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:52.046599 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:52.046608 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:52.047989 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:52.047993 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:52.048685 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:52.048695 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:52.049342 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:52.049347 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:52.049463 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:52.050399 (+ 936us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:52.050415 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:52.052835 (+ 2420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:52.055352 (+ 2517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:52.055368 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:52.055377 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:52.055715 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:52.056054 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:52.056057 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:52.056058 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:52.056278 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:52.056806 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:52.056814 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:52.056941 (+ 127us) client_negotiation.cc:769] Sending connection context
0806 11:36:52.057080 (+ 139us) client_negotiation.cc:240] Negotiation successful
0806 11:36:52.057361 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":103,"threads_started":1}
I20260806 11:36:52.062960 29543 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.21.14.254@KRBTEST.COM'} at 127.0.0.1:60580:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 11:36:52.063796 29543 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 11:36:52.070493 29837 tablet_service.cc:1467] Processing CreateTablet for tablet cf87eb6332f241bf8b98d416c5f1d313 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b57d67749a2f47ac8f30aac04693d690]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 11:36:52.070493 29690 tablet_service.cc:1467] Processing CreateTablet for tablet cf87eb6332f241bf8b98d416c5f1d313 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b57d67749a2f47ac8f30aac04693d690]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 11:36:52.070613 29995 tablet_service.cc:1467] Processing CreateTablet for tablet cf87eb6332f241bf8b98d416c5f1d313 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b57d67749a2f47ac8f30aac04693d690]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 11:36:52.071041 29837 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf87eb6332f241bf8b98d416c5f1d313. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:52.071038 29690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf87eb6332f241bf8b98d416c5f1d313. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:52.071053 29995 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf87eb6332f241bf8b98d416c5f1d313. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:52.075021 30101 tablet_bootstrap.cc:492] T cf87eb6332f241bf8b98d416c5f1d313 P 741edc4087df42139936dace73d5f68a: Bootstrap starting.
I20260806 11:36:52.075053 30100 tablet_bootstrap.cc:492] T cf87eb6332f241bf8b98d416c5f1d313 P 92712864d28043d5aa2771fbc08fc004: Bootstrap starting.
I20260806 11:36:52.075114 30102 tablet_bootstrap.cc:492] T cf87eb6332f241bf8b98d416c5f1d313 P f1d856374ce44857a183737b5e2b03ba: Bootstrap starting.
I20260806 11:36:52.076521 30101 tablet_bootstrap.cc:654] T cf87eb6332f241bf8b98d416c5f1d313 P 741edc4087df42139936dace73d5f68a: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:52.076507 30102 tablet_bootstrap.cc:654] T cf87eb6332f241bf8b98d416c5f1d313 P f1d856374ce44857a183737b5e2b03ba: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:52.076555 30100 tablet_bootstrap.cc:654] T cf87eb6332f241bf8b98d416c5f1d313 P 92712864d28043d5aa2771fbc08fc004: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:52.077978 30102 tablet_bootstrap.cc:492] T cf87eb6332f241bf8b98d416c5f1d313 P f1d856374ce44857a183737b5e2b03ba: No bootstrap required, opened a new log
I20260806 11:36:52.078028 30100 tablet_bootstrap.cc:492] T cf87eb6332f241bf8b98d416c5f1d313 P 92712864d28043d5aa2771fbc08fc004: No bootstrap required, opened a new log
I20260806 11:36:52.078097 30102 ts_tablet_manager.cc:1397] T cf87eb6332f241bf8b98d416c5f1d313 P f1d856374ce44857a183737b5e2b03ba: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.003s
I20260806 11:36:52.078148 30100 ts_tablet_manager.cc:1397] T cf87eb6332f241bf8b98d416c5f1d313 P 92712864d28043d5aa2771fbc08fc004: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.001s
I20260806 11:36:52.078230 30101 tablet_bootstrap.cc:492] T cf87eb6332f241bf8b98d416c5f1d313 P 741edc4087df42139936dace73d5f68a: No bootstrap required, opened a new log
I20260806 11:36:52.078341 30101 ts_tablet_manager.cc:1397] T cf87eb6332f241bf8b98d416c5f1d313 P 741edc4087df42139936dace73d5f68a: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20260806 11:36:52.078655 30100 raft_consensus.cc:348] T cf87eb6332f241bf8b98d416c5f1d313 P 92712864d28043d5aa2771fbc08fc004 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741edc4087df42139936dace73d5f68a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40189 } } peers { permanent_uuid: "92712864d28043d5aa2771fbc08fc004" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37297 } } peers { permanent_uuid: "f1d856374ce44857a183737b5e2b03ba" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38073 } }
I20260806 11:36:52.078632 30102 raft_consensus.cc:348] T cf87eb6332f241bf8b98d416c5f1d313 P f1d856374ce44857a183737b5e2b03ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741edc4087df42139936dace73d5f68a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40189 } } peers { permanent_uuid: "92712864d28043d5aa2771fbc08fc004" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37297 } } peers { permanent_uuid: "f1d856374ce44857a183737b5e2b03ba" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38073 } }
I20260806 11:36:52.078799 30102 raft_consensus.cc:374] T cf87eb6332f241bf8b98d416c5f1d313 P f1d856374ce44857a183737b5e2b03ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:52.078799 30100 raft_consensus.cc:374] T cf87eb6332f241bf8b98d416c5f1d313 P 92712864d28043d5aa2771fbc08fc004 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:52.078749 30101 raft_consensus.cc:348] T cf87eb6332f241bf8b98d416c5f1d313 P 741edc4087df42139936dace73d5f68a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741edc4087df42139936dace73d5f68a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40189 } } peers { permanent_uuid: "92712864d28043d5aa2771fbc08fc004" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37297 } } peers { permanent_uuid: "f1d856374ce44857a183737b5e2b03ba" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38073 } }
I20260806 11:36:52.078845 30100 raft_consensus.cc:729] T cf87eb6332f241bf8b98d416c5f1d313 P 92712864d28043d5aa2771fbc08fc004 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92712864d28043d5aa2771fbc08fc004, State: Initialized, Role: FOLLOWER
I20260806 11:36:52.078867 30102 raft_consensus.cc:729] T cf87eb6332f241bf8b98d416c5f1d313 P f1d856374ce44857a183737b5e2b03ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1d856374ce44857a183737b5e2b03ba, State: Initialized, Role: FOLLOWER
I20260806 11:36:52.078891 30101 raft_consensus.cc:374] T cf87eb6332f241bf8b98d416c5f1d313 P 741edc4087df42139936dace73d5f68a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:52.078929 30101 raft_consensus.cc:729] T cf87eb6332f241bf8b98d416c5f1d313 P 741edc4087df42139936dace73d5f68a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 741edc4087df42139936dace73d5f68a, State: Initialized, Role: FOLLOWER
I20260806 11:36:52.079030 30102 consensus_queue.cc:260] T cf87eb6332f241bf8b98d416c5f1d313 P f1d856374ce44857a183737b5e2b03ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741edc4087df42139936dace73d5f68a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40189 } } peers { permanent_uuid: "92712864d28043d5aa2771fbc08fc004" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37297 } } peers { permanent_uuid: "f1d856374ce44857a183737b5e2b03ba" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38073 } }
I20260806 11:36:52.079015 30100 consensus_queue.cc:260] T cf87eb6332f241bf8b98d416c5f1d313 P 92712864d28043d5aa2771fbc08fc004 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741edc4087df42139936dace73d5f68a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40189 } } peers { permanent_uuid: "92712864d28043d5aa2771fbc08fc004" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37297 } } peers { permanent_uuid: "f1d856374ce44857a183737b5e2b03ba" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38073 } }
I20260806 11:36:52.079140 30101 consensus_queue.cc:260] T cf87eb6332f241bf8b98d416c5f1d313 P 741edc4087df42139936dace73d5f68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741edc4087df42139936dace73d5f68a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40189 } } peers { permanent_uuid: "92712864d28043d5aa2771fbc08fc004" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37297 } } peers { permanent_uuid: "f1d856374ce44857a183737b5e2b03ba" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38073 } }
I20260806 11:36:52.079334 30102 ts_tablet_manager.cc:1428] T cf87eb6332f241bf8b98d416c5f1d313 P f1d856374ce44857a183737b5e2b03ba: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 11:36:52.079329 30100 ts_tablet_manager.cc:1428] T cf87eb6332f241bf8b98d416c5f1d313 P 92712864d28043d5aa2771fbc08fc004: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260806 11:36:52.079337 30106 tablet_replica.cc:439] T cf87eb6332f241bf8b98d416c5f1d313 P 92712864d28043d5aa2771fbc08fc004: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741edc4087df42139936dace73d5f68a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40189 } } peers { permanent_uuid: "92712864d28043d5aa2771fbc08fc004" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37297 } } peers { permanent_uuid: "f1d856374ce44857a183737b5e2b03ba" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38073 } } }
I20260806 11:36:52.079586 30106 tablet_replica.cc:442] T cf87eb6332f241bf8b98d416c5f1d313 P 92712864d28043d5aa2771fbc08fc004: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 11:36:52.079535 30108 tablet_replica.cc:439] T cf87eb6332f241bf8b98d416c5f1d313 P 741edc4087df42139936dace73d5f68a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741edc4087df42139936dace73d5f68a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40189 } } peers { permanent_uuid: "92712864d28043d5aa2771fbc08fc004" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37297 } } peers { permanent_uuid: "f1d856374ce44857a183737b5e2b03ba" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38073 } } }
I20260806 11:36:52.079548 30107 tablet_replica.cc:439] T cf87eb6332f241bf8b98d416c5f1d313 P f1d856374ce44857a183737b5e2b03ba: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741edc4087df42139936dace73d5f68a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40189 } } peers { permanent_uuid: "92712864d28043d5aa2771fbc08fc004" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37297 } } peers { permanent_uuid: "f1d856374ce44857a183737b5e2b03ba" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38073 } } }
I20260806 11:36:52.079689 30108 tablet_replica.cc:442] T cf87eb6332f241bf8b98d416c5f1d313 P 741edc4087df42139936dace73d5f68a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 11:36:52.079694 30107 tablet_replica.cc:442] T cf87eb6332f241bf8b98d416c5f1d313 P f1d856374ce44857a183737b5e2b03ba: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 11:36:52.079970 30101 ts_tablet_manager.cc:1428] T cf87eb6332f241bf8b98d416c5f1d313 P 741edc4087df42139936dace73d5f68a: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
W20260806 11:36:52.085558 29760 tablet.cc:2369] T cf87eb6332f241bf8b98d416c5f1d313 P 741edc4087df42139936dace73d5f68a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 11:36:52.090798 29911 tablet.cc:2369] T cf87eb6332f241bf8b98d416c5f1d313 P 92712864d28043d5aa2771fbc08fc004: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 11:36:52.101778 30070 tablet.cc:2369] T cf87eb6332f241bf8b98d416c5f1d313 P f1d856374ce44857a183737b5e2b03ba: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 11:36:52.109292 30108 raft_consensus.cc:482] T cf87eb6332f241bf8b98d416c5f1d313 P 741edc4087df42139936dace73d5f68a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 11:36:52.109473 30108 raft_consensus.cc:504] T cf87eb6332f241bf8b98d416c5f1d313 P 741edc4087df42139936dace73d5f68a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741edc4087df42139936dace73d5f68a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40189 } } peers { permanent_uuid: "92712864d28043d5aa2771fbc08fc004" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37297 } } peers { permanent_uuid: "f1d856374ce44857a183737b5e2b03ba" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38073 } }
I20260806 11:36:52.109934 30108 leader_election.cc:288] T cf87eb6332f241bf8b98d416c5f1d313 P 741edc4087df42139936dace73d5f68a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 92712864d28043d5aa2771fbc08fc004 (127.21.14.194:37297), f1d856374ce44857a183737b5e2b03ba (127.21.14.195:38073)
I20260806 11:36:52.110641 30018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf87eb6332f241bf8b98d416c5f1d313" candidate_uuid: "741edc4087df42139936dace73d5f68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1d856374ce44857a183737b5e2b03ba" is_pre_election: true
I20260806 11:36:52.110692 29859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf87eb6332f241bf8b98d416c5f1d313" candidate_uuid: "741edc4087df42139936dace73d5f68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92712864d28043d5aa2771fbc08fc004" is_pre_election: true
I20260806 11:36:52.110839 30018 raft_consensus.cc:2454] T cf87eb6332f241bf8b98d416c5f1d313 P f1d856374ce44857a183737b5e2b03ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 741edc4087df42139936dace73d5f68a in term 0.
I20260806 11:36:52.110899 29859 raft_consensus.cc:2454] T cf87eb6332f241bf8b98d416c5f1d313 P 92712864d28043d5aa2771fbc08fc004 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 741edc4087df42139936dace73d5f68a in term 0.
I20260806 11:36:52.111397 29640 leader_election.cc:302] T cf87eb6332f241bf8b98d416c5f1d313 P 741edc4087df42139936dace73d5f68a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 741edc4087df42139936dace73d5f68a, f1d856374ce44857a183737b5e2b03ba; no voters:
I20260806 11:36:52.111682 30108 raft_consensus.cc:2781] T cf87eb6332f241bf8b98d416c5f1d313 P 741edc4087df42139936dace73d5f68a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 11:36:52.111874 30108 raft_consensus.cc:482] T cf87eb6332f241bf8b98d416c5f1d313 P 741edc4087df42139936dace73d5f68a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 11:36:52.111953 30108 raft_consensus.cc:3037] T cf87eb6332f241bf8b98d416c5f1d313 P 741edc4087df42139936dace73d5f68a [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:36:52.113438 30108 raft_consensus.cc:504] T cf87eb6332f241bf8b98d416c5f1d313 P 741edc4087df42139936dace73d5f68a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741edc4087df42139936dace73d5f68a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40189 } } peers { permanent_uuid: "92712864d28043d5aa2771fbc08fc004" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37297 } } peers { permanent_uuid: "f1d856374ce44857a183737b5e2b03ba" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38073 } }
I20260806 11:36:52.114089 30108 leader_election.cc:288] T cf87eb6332f241bf8b98d416c5f1d313 P 741edc4087df42139936dace73d5f68a [CANDIDATE]: Term 1 election: Requested vote from peers 92712864d28043d5aa2771fbc08fc004 (127.21.14.194:37297), f1d856374ce44857a183737b5e2b03ba (127.21.14.195:38073)
I20260806 11:36:52.114655 29859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf87eb6332f241bf8b98d416c5f1d313" candidate_uuid: "741edc4087df42139936dace73d5f68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92712864d28043d5aa2771fbc08fc004"
I20260806 11:36:52.114836 29859 raft_consensus.cc:3037] T cf87eb6332f241bf8b98d416c5f1d313 P 92712864d28043d5aa2771fbc08fc004 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:36:52.114782 30018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf87eb6332f241bf8b98d416c5f1d313" candidate_uuid: "741edc4087df42139936dace73d5f68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1d856374ce44857a183737b5e2b03ba"
I20260806 11:36:52.114991 30018 raft_consensus.cc:3037] T cf87eb6332f241bf8b98d416c5f1d313 P f1d856374ce44857a183737b5e2b03ba [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:36:52.116114 30018 raft_consensus.cc:2454] T cf87eb6332f241bf8b98d416c5f1d313 P f1d856374ce44857a183737b5e2b03ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 741edc4087df42139936dace73d5f68a in term 1.
I20260806 11:36:52.116492 29859 raft_consensus.cc:2454] T cf87eb6332f241bf8b98d416c5f1d313 P 92712864d28043d5aa2771fbc08fc004 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 741edc4087df42139936dace73d5f68a in term 1.
I20260806 11:36:52.116557 29640 leader_election.cc:302] T cf87eb6332f241bf8b98d416c5f1d313 P 741edc4087df42139936dace73d5f68a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 741edc4087df42139936dace73d5f68a, f1d856374ce44857a183737b5e2b03ba; no voters:
I20260806 11:36:52.116834 30108 raft_consensus.cc:2781] T cf87eb6332f241bf8b98d416c5f1d313 P 741edc4087df42139936dace73d5f68a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:36:52.116962 30108 raft_consensus.cc:686] T cf87eb6332f241bf8b98d416c5f1d313 P 741edc4087df42139936dace73d5f68a [term 1 LEADER]: Becoming Leader. State: Replica: 741edc4087df42139936dace73d5f68a, State: Running, Role: LEADER
I20260806 11:36:52.117136 30108 consensus_queue.cc:237] T cf87eb6332f241bf8b98d416c5f1d313 P 741edc4087df42139936dace73d5f68a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741edc4087df42139936dace73d5f68a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40189 } } peers { permanent_uuid: "92712864d28043d5aa2771fbc08fc004" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37297 } } peers { permanent_uuid: "f1d856374ce44857a183737b5e2b03ba" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38073 } }
I20260806 11:36:52.117888 30119 tablet_replica.cc:439] T cf87eb6332f241bf8b98d416c5f1d313 P 741edc4087df42139936dace73d5f68a: TxnStatusTablet state changed. Reason: New leader 741edc4087df42139936dace73d5f68a. Latest consensus state: current_term: 1 leader_uuid: "741edc4087df42139936dace73d5f68a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741edc4087df42139936dace73d5f68a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40189 } } peers { permanent_uuid: "92712864d28043d5aa2771fbc08fc004" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37297 } } peers { permanent_uuid: "f1d856374ce44857a183737b5e2b03ba" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38073 } } }
I20260806 11:36:52.118034 30119 tablet_replica.cc:442] T cf87eb6332f241bf8b98d416c5f1d313 P 741edc4087df42139936dace73d5f68a: This TxnStatusTablet replica's current role is: LEADER
I20260806 11:36:52.118382 30145 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 11:36:52.118568 30145 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 11:36:52.119410 29542 catalog_manager.cc:5495] T cf87eb6332f241bf8b98d416c5f1d313 P 741edc4087df42139936dace73d5f68a reported cstate change: term changed from 0 to 1, leader changed from <none> to 741edc4087df42139936dace73d5f68a (127.21.14.193). New cstate: current_term: 1 leader_uuid: "741edc4087df42139936dace73d5f68a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741edc4087df42139936dace73d5f68a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92712864d28043d5aa2771fbc08fc004" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37297 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f1d856374ce44857a183737b5e2b03ba" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38073 } health_report { overall_health: UNKNOWN } } }
I20260806 11:36:52.138648 30139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:52.133795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.193:40189 (local address 127.0.0.1:34202)
0806 11:36:52.134051 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:52.134066 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:52.134323 (+ 257us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:52.134764 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:52.134766 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:52.134775 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 11:36:52.134973 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:52.134979 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:52.136318 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:52.136322 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:52.137035 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:52.137046 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:52.137743 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:52.137746 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:52.137831 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:52.137903 (+ 72us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 11:36:52.138309 (+ 406us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 11:36:52.138328 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 11:36:52.138400 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 11:36:52.138505 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106}
I20260806 11:36:52.138662 30098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:52.133960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34202 (local address 127.21.14.193:40189)
0806 11:36:52.134113 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:52.134118 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:52.134144 (+ 26us) server_negotiation.cc:407] Connection header received
0806 11:36:52.134523 (+ 379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:52.134526 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:52.134587 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:52.134685 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 11:36:52.135150 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:52.135990 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:52.137203 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:52.137573 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:52.137706 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:52.138074 (+ 368us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 11:36:52.138185 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 11:36:52.138270 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:52.138436 (+ 166us) server_negotiation.cc:299] Negotiation successful
0806 11:36:52.138509 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 11:36:52.141981 30018 raft_consensus.cc:1261] T cf87eb6332f241bf8b98d416c5f1d313 P f1d856374ce44857a183737b5e2b03ba [term 1 FOLLOWER]: Refusing update from remote peer 741edc4087df42139936dace73d5f68a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 11:36:52.142118 29859 raft_consensus.cc:1261] T cf87eb6332f241bf8b98d416c5f1d313 P 92712864d28043d5aa2771fbc08fc004 [term 1 FOLLOWER]: Refusing update from remote peer 741edc4087df42139936dace73d5f68a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 11:36:52.142635 30119 consensus_queue.cc:1035] T cf87eb6332f241bf8b98d416c5f1d313 P 741edc4087df42139936dace73d5f68a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1d856374ce44857a183737b5e2b03ba" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38073 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 11:36:52.143107 30108 consensus_queue.cc:1035] T cf87eb6332f241bf8b98d416c5f1d313 P 741edc4087df42139936dace73d5f68a [LEADER]: Connected to new peer: Peer: permanent_uuid: "92712864d28043d5aa2771fbc08fc004" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 11:36:52.144629 30107 tablet_replica.cc:439] T cf87eb6332f241bf8b98d416c5f1d313 P f1d856374ce44857a183737b5e2b03ba: TxnStatusTablet state changed. Reason: New leader 741edc4087df42139936dace73d5f68a. Latest consensus state: current_term: 1 leader_uuid: "741edc4087df42139936dace73d5f68a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741edc4087df42139936dace73d5f68a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40189 } } peers { permanent_uuid: "92712864d28043d5aa2771fbc08fc004" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37297 } } peers { permanent_uuid: "f1d856374ce44857a183737b5e2b03ba" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38073 } } }
I20260806 11:36:52.144788 30107 tablet_replica.cc:442] T cf87eb6332f241bf8b98d416c5f1d313 P f1d856374ce44857a183737b5e2b03ba: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 11:36:52.144876 30106 tablet_replica.cc:439] T cf87eb6332f241bf8b98d416c5f1d313 P 92712864d28043d5aa2771fbc08fc004: TxnStatusTablet state changed. Reason: New leader 741edc4087df42139936dace73d5f68a. Latest consensus state: current_term: 1 leader_uuid: "741edc4087df42139936dace73d5f68a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741edc4087df42139936dace73d5f68a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40189 } } peers { permanent_uuid: "92712864d28043d5aa2771fbc08fc004" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37297 } } peers { permanent_uuid: "f1d856374ce44857a183737b5e2b03ba" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38073 } } }
I20260806 11:36:52.145001 30106 tablet_replica.cc:442] T cf87eb6332f241bf8b98d416c5f1d313 P 92712864d28043d5aa2771fbc08fc004: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 11:36:52.147953 30107 tablet_replica.cc:439] T cf87eb6332f241bf8b98d416c5f1d313 P f1d856374ce44857a183737b5e2b03ba: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "741edc4087df42139936dace73d5f68a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741edc4087df42139936dace73d5f68a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40189 } } peers { permanent_uuid: "92712864d28043d5aa2771fbc08fc004" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37297 } } peers { permanent_uuid: "f1d856374ce44857a183737b5e2b03ba" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38073 } } }
I20260806 11:36:52.148140 30107 tablet_replica.cc:442] T cf87eb6332f241bf8b98d416c5f1d313 P f1d856374ce44857a183737b5e2b03ba: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 11:36:52.148448 30108 tablet_replica.cc:439] T cf87eb6332f241bf8b98d416c5f1d313 P 741edc4087df42139936dace73d5f68a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "741edc4087df42139936dace73d5f68a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741edc4087df42139936dace73d5f68a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40189 } } peers { permanent_uuid: "92712864d28043d5aa2771fbc08fc004" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37297 } } peers { permanent_uuid: "f1d856374ce44857a183737b5e2b03ba" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38073 } } }
I20260806 11:36:52.148576 30108 tablet_replica.cc:442] T cf87eb6332f241bf8b98d416c5f1d313 P 741edc4087df42139936dace73d5f68a: This TxnStatusTablet replica's current role is: LEADER
I20260806 11:36:52.148782 30108 tablet_replica.cc:439] T cf87eb6332f241bf8b98d416c5f1d313 P 741edc4087df42139936dace73d5f68a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "741edc4087df42139936dace73d5f68a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741edc4087df42139936dace73d5f68a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40189 } } peers { permanent_uuid: "92712864d28043d5aa2771fbc08fc004" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37297 } } peers { permanent_uuid: "f1d856374ce44857a183737b5e2b03ba" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38073 } } }
I20260806 11:36:52.148872 30108 tablet_replica.cc:442] T cf87eb6332f241bf8b98d416c5f1d313 P 741edc4087df42139936dace73d5f68a: This TxnStatusTablet replica's current role is: LEADER
I20260806 11:36:52.149098 30106 tablet_replica.cc:439] T cf87eb6332f241bf8b98d416c5f1d313 P 92712864d28043d5aa2771fbc08fc004: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "741edc4087df42139936dace73d5f68a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741edc4087df42139936dace73d5f68a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40189 } } peers { permanent_uuid: "92712864d28043d5aa2771fbc08fc004" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37297 } } peers { permanent_uuid: "f1d856374ce44857a183737b5e2b03ba" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38073 } } }
I20260806 11:36:52.149237 30106 tablet_replica.cc:442] T cf87eb6332f241bf8b98d416c5f1d313 P 92712864d28043d5aa2771fbc08fc004: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 11:36:52.149708 30114 mvcc.cc:205] Tried to move back new op lower bound from 7315522404927918080 to 7315522404833415168. Current Snapshot: MvccSnapshot[applied={T|T < 7315522404927918080 or (T in {7315522404927918080})}]
I20260806 11:36:52.161715 30095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:52.156229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60306 (local address 127.21.14.194:37297)
0806 11:36:52.156395 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:52.156403 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:52.156622 (+ 219us) server_negotiation.cc:407] Connection header received
0806 11:36:52.156808 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:52.156812 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:52.156893 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:52.157038 (+ 145us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 11:36:52.157522 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:52.158332 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:52.159502 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:52.159985 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:52.160133 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:52.160472 (+ 339us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 11:36:52.161146 (+ 674us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 11:36:52.161274 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:52.161448 (+ 174us) server_negotiation.cc:299] Negotiation successful
0806 11:36:52.161527 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 11:36:52.177399 30098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:52.171870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:34133 (local address 127.21.14.193:40189)
0806 11:36:52.172028 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:52.172034 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:52.172648 (+ 614us) server_negotiation.cc:407] Connection header received
0806 11:36:52.172716 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:52.172719 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:52.172773 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:52.172852 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 11:36:52.173300 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:52.174097 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:52.175829 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:52.176189 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:52.176295 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:52.176785 (+ 490us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 11:36:52.176883 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 11:36:52.176956 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:52.177172 (+ 216us) server_negotiation.cc:299] Negotiation successful
0806 11:36:52.177264 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 11:36:52.177395 30155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:52.171805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.193:40189 (local address 127.21.14.194:34133)
0806 11:36:52.172417 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:52.172434 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:52.172580 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:52.172889 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:52.172892 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:52.172900 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 11:36:52.173049 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:52.173055 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:52.174237 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:52.174242 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:52.175473 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:52.175492 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:52.176346 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:52.176350 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:52.176484 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:52.176566 (+ 82us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 11:36:52.176991 (+ 425us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 11:36:52.176998 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 11:36:52.177077 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 11:36:52.177187 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":147,"threads_started":1}
I20260806 11:36:52.187346 30018 raft_consensus.cc:1261] T fe2f39af2fd1453597dab9175511b94c P f1d856374ce44857a183737b5e2b03ba [term 1 FOLLOWER]: Refusing update from remote peer 92712864d28043d5aa2771fbc08fc004: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 11:36:52.187438 29710 raft_consensus.cc:1261] T fe2f39af2fd1453597dab9175511b94c P 741edc4087df42139936dace73d5f68a [term 1 FOLLOWER]: Refusing update from remote peer 92712864d28043d5aa2771fbc08fc004: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 11:36:52.188759 30132 consensus_queue.cc:1035] T fe2f39af2fd1453597dab9175511b94c P 92712864d28043d5aa2771fbc08fc004 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1d856374ce44857a183737b5e2b03ba" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38073 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260806 11:36:52.189153 30106 consensus_queue.cc:1035] T fe2f39af2fd1453597dab9175511b94c P 92712864d28043d5aa2771fbc08fc004 [LEADER]: Connected to new peer: Peer: permanent_uuid: "741edc4087df42139936dace73d5f68a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 11:36:52.217940 30095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:52.213593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:39337 (local address 127.21.14.194:37297)
0806 11:36:52.213754 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:52.213759 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:52.214443 (+ 684us) server_negotiation.cc:407] Connection header received
0806 11:36:52.214500 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:52.214502 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:52.214555 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:52.214654 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 11:36:52.214985 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:52.215562 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:52.216699 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:52.216935 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:52.217060 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:52.217417 (+ 357us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 11:36:52.217533 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 11:36:52.217622 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:52.217716 (+ 94us) server_negotiation.cc:299] Negotiation successful
0806 11:36:52.217779 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 11:36:52.217989 30159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:52.213432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.194:37297 (local address 127.21.14.193:39337)
0806 11:36:52.213938 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:52.213955 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:52.214104 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:52.214668 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:52.214672 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:52.214681 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 11:36:52.214803 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:52.214810 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:52.215685 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:52.215689 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:52.216534 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:52.216548 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:52.217068 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:52.217072 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:52.217183 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:52.217250 (+ 67us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 11:36:52.217628 (+ 378us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 11:36:52.217638 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 11:36:52.217707 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 11:36:52.217806 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":124,"threads_started":1}
I20260806 11:36:52.253465 29539 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60574:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 11:36:52.253683 29539 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60574:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 11:36:52.256465 29539 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ed204855a8dd4cd686fd550178b9563e: Sending DeleteTablet for 3 replicas of tablet fe2f39af2fd1453597dab9175511b94c
I20260806 11:36:52.257244 29690 tablet_service.cc:1514] Processing DeleteTablet for tablet fe2f39af2fd1453597dab9175511b94c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:36:52 UTC) from {username='oryx', principal='oryx/127.21.14.254@KRBTEST.COM'} at 127.0.0.1:34180
I20260806 11:36:52.257371 29837 tablet_service.cc:1514] Processing DeleteTablet for tablet fe2f39af2fd1453597dab9175511b94c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:36:52 UTC) from {username='oryx', principal='oryx/127.21.14.254@KRBTEST.COM'} at 127.0.0.1:60302
I20260806 11:36:52.257470 30125 tablet_replica.cc:330] T fe2f39af2fd1453597dab9175511b94c P 741edc4087df42139936dace73d5f68a: stopping tablet replica
I20260806 11:36:52.257563 30124 tablet_replica.cc:330] T fe2f39af2fd1453597dab9175511b94c P 92712864d28043d5aa2771fbc08fc004: stopping tablet replica
I20260806 11:36:52.257611 30125 raft_consensus.cc:2229] T fe2f39af2fd1453597dab9175511b94c P 741edc4087df42139936dace73d5f68a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 11:36:52.257678 30124 raft_consensus.cc:2229] T fe2f39af2fd1453597dab9175511b94c P 92712864d28043d5aa2771fbc08fc004 [term 1 LEADER]: Raft consensus shutting down.
I20260806 11:36:52.257772 30125 raft_consensus.cc:2258] T fe2f39af2fd1453597dab9175511b94c P 741edc4087df42139936dace73d5f68a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 11:36:52.258077 30124 raft_consensus.cc:2258] T fe2f39af2fd1453597dab9175511b94c P 92712864d28043d5aa2771fbc08fc004 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 11:36:52.258616 29995 tablet_service.cc:1514] Processing DeleteTablet for tablet fe2f39af2fd1453597dab9175511b94c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:36:52 UTC) from {username='oryx', principal='oryx/127.21.14.254@KRBTEST.COM'} at 127.0.0.1:46500
I20260806 11:36:52.258812 30126 tablet_replica.cc:330] T fe2f39af2fd1453597dab9175511b94c P f1d856374ce44857a183737b5e2b03ba: stopping tablet replica
I20260806 11:36:52.258976 30126 raft_consensus.cc:2229] T fe2f39af2fd1453597dab9175511b94c P f1d856374ce44857a183737b5e2b03ba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 11:36:52.258965 30125 ts_tablet_manager.cc:1905] T fe2f39af2fd1453597dab9175511b94c P 741edc4087df42139936dace73d5f68a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 11:36:52.259102 30126 raft_consensus.cc:2258] T fe2f39af2fd1453597dab9175511b94c P f1d856374ce44857a183737b5e2b03ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 11:36:52.260012 30126 ts_tablet_manager.cc:1905] T fe2f39af2fd1453597dab9175511b94c P f1d856374ce44857a183737b5e2b03ba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 11:36:52.262825 30125 ts_tablet_manager.cc:1918] T fe2f39af2fd1453597dab9175511b94c P 741edc4087df42139936dace73d5f68a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 11:36:52.263193 30125 log.cc:1196] T fe2f39af2fd1453597dab9175511b94c P 741edc4087df42139936dace73d5f68a: Deleting WAL directory at /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/ts-0/wal/wals/fe2f39af2fd1453597dab9175511b94c
I20260806 11:36:52.263424 30126 ts_tablet_manager.cc:1918] T fe2f39af2fd1453597dab9175511b94c P f1d856374ce44857a183737b5e2b03ba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 11:36:52.263533 30126 log.cc:1196] T fe2f39af2fd1453597dab9175511b94c P f1d856374ce44857a183737b5e2b03ba: Deleting WAL directory at /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/ts-2/wal/wals/fe2f39af2fd1453597dab9175511b94c
I20260806 11:36:52.263832 30126 ts_tablet_manager.cc:1939] T fe2f39af2fd1453597dab9175511b94c P f1d856374ce44857a183737b5e2b03ba: Deleting consensus metadata
I20260806 11:36:52.264510 30125 ts_tablet_manager.cc:1939] T fe2f39af2fd1453597dab9175511b94c P 741edc4087df42139936dace73d5f68a: Deleting consensus metadata
I20260806 11:36:52.265388 29527 catalog_manager.cc:4841] TS 741edc4087df42139936dace73d5f68a (127.21.14.193:40189): tablet fe2f39af2fd1453597dab9175511b94c (table test-table [id=4d97ed31ab1349eab491472ed191a36f]) successfully deleted
I20260806 11:36:52.265530 30124 ts_tablet_manager.cc:1905] T fe2f39af2fd1453597dab9175511b94c P 92712864d28043d5aa2771fbc08fc004: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 11:36:52.269336 30124 ts_tablet_manager.cc:1918] T fe2f39af2fd1453597dab9175511b94c P 92712864d28043d5aa2771fbc08fc004: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 11:36:52.269474 30124 log.cc:1196] T fe2f39af2fd1453597dab9175511b94c P 92712864d28043d5aa2771fbc08fc004: Deleting WAL directory at /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/ts-1/wal/wals/fe2f39af2fd1453597dab9175511b94c
I20260806 11:36:52.269796 30124 ts_tablet_manager.cc:1939] T fe2f39af2fd1453597dab9175511b94c P 92712864d28043d5aa2771fbc08fc004: Deleting consensus metadata
I20260806 11:36:52.270051 29530 catalog_manager.cc:4841] TS f1d856374ce44857a183737b5e2b03ba (127.21.14.195:38073): tablet fe2f39af2fd1453597dab9175511b94c (table test-table [id=4d97ed31ab1349eab491472ed191a36f]) successfully deleted
I20260806 11:36:52.273723 29528 catalog_manager.cc:4841] TS 92712864d28043d5aa2771fbc08fc004 (127.21.14.194:37297): tablet fe2f39af2fd1453597dab9175511b94c (table test-table [id=4d97ed31ab1349eab491472ed191a36f]) successfully deleted
Aug 06 11:36:52 dist-test-slave-7c35 krb5kdc[29497](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016212, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 11:36:52 dist-test-slave-7c35 krb5kdc[29497](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 11:36:52 dist-test-slave-7c35 krb5kdc[29497](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
W20260806 11:36:52.309128 29596 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:36:52.300086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60584 (local address 127.21.14.254:35125)
0806 11:36:52.300238 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:52.300243 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:52.300608 (+ 365us) server_negotiation.cc:407] Connection header received
0806 11:36:52.300770 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:52.300774 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:52.300835 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:52.300935 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:52.301969 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:52.302804 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:52.304095 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:52.304442 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:52.304556 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:52.308979 (+ 4423us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60584: BlockingRecv error: recv got EOF from 127.0.0.1:60584 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 06 11:36:52 dist-test-slave-7c35 krb5kdc[29497](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016212, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM
I20260806 11:36:52.322216 29596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:52.311508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60590 (local address 127.21.14.254:35125)
0806 11:36:52.311649 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:52.311654 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:52.311955 (+ 301us) server_negotiation.cc:407] Connection header received
0806 11:36:52.312090 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:52.312093 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:52.312150 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:52.312243 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:52.313112 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:52.313825 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:52.315141 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:52.315464 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:52.315578 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:52.318635 (+ 3057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:52.318639 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:52.318642 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:52.318678 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:52.320302 (+ 1624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:52.320819 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:52.320823 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:52.320825 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:52.320915 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:52.321220 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:52.321225 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:52.321227 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:52.321448 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:52.321598 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:52.321870 (+ 272us) server_negotiation.cc:299] Negotiation successful
0806 11:36:52.322021 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 11:36:52.325492 29539 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60590:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 11:36:52.326015 29539 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 11:36:52.332235 29690 tablet_service.cc:1467] Processing CreateTablet for tablet 27d3535d45784e539e95fba5f0441b26 (DEFAULT_TABLE table=test-table [id=fec607f4a843415f9a58d9a5c2926cb3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:36:52.332306 29837 tablet_service.cc:1467] Processing CreateTablet for tablet 27d3535d45784e539e95fba5f0441b26 (DEFAULT_TABLE table=test-table [id=fec607f4a843415f9a58d9a5c2926cb3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:36:52.332629 29837 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27d3535d45784e539e95fba5f0441b26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:52.332633 29690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27d3535d45784e539e95fba5f0441b26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:52.332620 29995 tablet_service.cc:1467] Processing CreateTablet for tablet 27d3535d45784e539e95fba5f0441b26 (DEFAULT_TABLE table=test-table [id=fec607f4a843415f9a58d9a5c2926cb3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:36:52.332916 29995 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27d3535d45784e539e95fba5f0441b26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:52.335495 30100 tablet_bootstrap.cc:492] T 27d3535d45784e539e95fba5f0441b26 P 92712864d28043d5aa2771fbc08fc004: Bootstrap starting.
I20260806 11:36:52.335520 30101 tablet_bootstrap.cc:492] T 27d3535d45784e539e95fba5f0441b26 P 741edc4087df42139936dace73d5f68a: Bootstrap starting.
I20260806 11:36:52.336220 30102 tablet_bootstrap.cc:492] T 27d3535d45784e539e95fba5f0441b26 P f1d856374ce44857a183737b5e2b03ba: Bootstrap starting.
I20260806 11:36:52.336591 30100 tablet_bootstrap.cc:654] T 27d3535d45784e539e95fba5f0441b26 P 92712864d28043d5aa2771fbc08fc004: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:52.336752 30101 tablet_bootstrap.cc:654] T 27d3535d45784e539e95fba5f0441b26 P 741edc4087df42139936dace73d5f68a: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:52.337221 30102 tablet_bootstrap.cc:654] T 27d3535d45784e539e95fba5f0441b26 P f1d856374ce44857a183737b5e2b03ba: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:52.338935 30100 tablet_bootstrap.cc:492] T 27d3535d45784e539e95fba5f0441b26 P 92712864d28043d5aa2771fbc08fc004: No bootstrap required, opened a new log
I20260806 11:36:52.339043 30100 ts_tablet_manager.cc:1397] T 27d3535d45784e539e95fba5f0441b26 P 92712864d28043d5aa2771fbc08fc004: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.002s
I20260806 11:36:52.339190 30102 tablet_bootstrap.cc:492] T 27d3535d45784e539e95fba5f0441b26 P f1d856374ce44857a183737b5e2b03ba: No bootstrap required, opened a new log
I20260806 11:36:52.339277 30102 ts_tablet_manager.cc:1397] T 27d3535d45784e539e95fba5f0441b26 P f1d856374ce44857a183737b5e2b03ba: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 11:36:52.339519 30100 raft_consensus.cc:348] T 27d3535d45784e539e95fba5f0441b26 P 92712864d28043d5aa2771fbc08fc004 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741edc4087df42139936dace73d5f68a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40189 } } peers { permanent_uuid: "92712864d28043d5aa2771fbc08fc004" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37297 } } peers { permanent_uuid: "f1d856374ce44857a183737b5e2b03ba" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38073 } }
I20260806 11:36:52.339630 30100 raft_consensus.cc:374] T 27d3535d45784e539e95fba5f0441b26 P 92712864d28043d5aa2771fbc08fc004 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:52.339661 30100 raft_consensus.cc:729] T 27d3535d45784e539e95fba5f0441b26 P 92712864d28043d5aa2771fbc08fc004 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92712864d28043d5aa2771fbc08fc004, State: Initialized, Role: FOLLOWER
I20260806 11:36:52.339792 30100 consensus_queue.cc:260] T 27d3535d45784e539e95fba5f0441b26 P 92712864d28043d5aa2771fbc08fc004 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741edc4087df42139936dace73d5f68a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40189 } } peers { permanent_uuid: "92712864d28043d5aa2771fbc08fc004" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37297 } } peers { permanent_uuid: "f1d856374ce44857a183737b5e2b03ba" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38073 } }
I20260806 11:36:52.339735 30102 raft_consensus.cc:348] T 27d3535d45784e539e95fba5f0441b26 P f1d856374ce44857a183737b5e2b03ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741edc4087df42139936dace73d5f68a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40189 } } peers { permanent_uuid: "92712864d28043d5aa2771fbc08fc004" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37297 } } peers { permanent_uuid: "f1d856374ce44857a183737b5e2b03ba" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38073 } }
I20260806 11:36:52.339890 30102 raft_consensus.cc:374] T 27d3535d45784e539e95fba5f0441b26 P f1d856374ce44857a183737b5e2b03ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:52.339932 30102 raft_consensus.cc:729] T 27d3535d45784e539e95fba5f0441b26 P f1d856374ce44857a183737b5e2b03ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1d856374ce44857a183737b5e2b03ba, State: Initialized, Role: FOLLOWER
I20260806 11:36:52.340054 30100 ts_tablet_manager.cc:1428] T 27d3535d45784e539e95fba5f0441b26 P 92712864d28043d5aa2771fbc08fc004: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 11:36:52.340103 30102 consensus_queue.cc:260] T 27d3535d45784e539e95fba5f0441b26 P f1d856374ce44857a183737b5e2b03ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741edc4087df42139936dace73d5f68a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40189 } } peers { permanent_uuid: "92712864d28043d5aa2771fbc08fc004" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37297 } } peers { permanent_uuid: "f1d856374ce44857a183737b5e2b03ba" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38073 } }
I20260806 11:36:52.340340 30102 ts_tablet_manager.cc:1428] T 27d3535d45784e539e95fba5f0441b26 P f1d856374ce44857a183737b5e2b03ba: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 11:36:52.343639 30101 tablet_bootstrap.cc:492] T 27d3535d45784e539e95fba5f0441b26 P 741edc4087df42139936dace73d5f68a: No bootstrap required, opened a new log
I20260806 11:36:52.343773 30101 ts_tablet_manager.cc:1397] T 27d3535d45784e539e95fba5f0441b26 P 741edc4087df42139936dace73d5f68a: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20260806 11:36:52.344288 30101 raft_consensus.cc:348] T 27d3535d45784e539e95fba5f0441b26 P 741edc4087df42139936dace73d5f68a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741edc4087df42139936dace73d5f68a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40189 } } peers { permanent_uuid: "92712864d28043d5aa2771fbc08fc004" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37297 } } peers { permanent_uuid: "f1d856374ce44857a183737b5e2b03ba" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38073 } }
I20260806 11:36:52.344444 30101 raft_consensus.cc:374] T 27d3535d45784e539e95fba5f0441b26 P 741edc4087df42139936dace73d5f68a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:52.344485 30101 raft_consensus.cc:729] T 27d3535d45784e539e95fba5f0441b26 P 741edc4087df42139936dace73d5f68a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 741edc4087df42139936dace73d5f68a, State: Initialized, Role: FOLLOWER
I20260806 11:36:52.348266 30101 consensus_queue.cc:260] T 27d3535d45784e539e95fba5f0441b26 P 741edc4087df42139936dace73d5f68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741edc4087df42139936dace73d5f68a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40189 } } peers { permanent_uuid: "92712864d28043d5aa2771fbc08fc004" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37297 } } peers { permanent_uuid: "f1d856374ce44857a183737b5e2b03ba" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38073 } }
I20260806 11:36:52.349233 30101 ts_tablet_manager.cc:1428] T 27d3535d45784e539e95fba5f0441b26 P 741edc4087df42139936dace73d5f68a: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20260806 11:36:52.388428 30107 raft_consensus.cc:482] T 27d3535d45784e539e95fba5f0441b26 P f1d856374ce44857a183737b5e2b03ba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 11:36:52.388640 30107 raft_consensus.cc:504] T 27d3535d45784e539e95fba5f0441b26 P f1d856374ce44857a183737b5e2b03ba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741edc4087df42139936dace73d5f68a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40189 } } peers { permanent_uuid: "92712864d28043d5aa2771fbc08fc004" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37297 } } peers { permanent_uuid: "f1d856374ce44857a183737b5e2b03ba" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38073 } }
I20260806 11:36:52.389961 30107 leader_election.cc:288] T 27d3535d45784e539e95fba5f0441b26 P f1d856374ce44857a183737b5e2b03ba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 741edc4087df42139936dace73d5f68a (127.21.14.193:40189), 92712864d28043d5aa2771fbc08fc004 (127.21.14.194:37297)
I20260806 11:36:52.396328 30095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:52.390377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:41423 (local address 127.21.14.194:37297)
0806 11:36:52.390517 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:52.390523 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:52.390721 (+ 198us) server_negotiation.cc:407] Connection header received
0806 11:36:52.390958 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:52.390962 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:52.391018 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:52.391112 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:36:52.391958 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:52.392462 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:52.394485 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:52.395207 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:52.395325 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:52.395383 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:52.396078 (+ 695us) server_negotiation.cc:299] Negotiation successful
0806 11:36:52.396177 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 11:36:52.396435 30179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:52.390235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.194:37297 (local address 127.21.14.195:41423)
0806 11:36:52.390610 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:52.390623 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:52.390776 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:52.391139 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:52.391144 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:52.391165 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:36:52.391821 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:52.391829 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:52.392593 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:52.392598 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:52.394282 (+ 1684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:52.394299 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:52.395529 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:52.395532 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:52.395761 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:52.395829 (+ 68us) client_negotiation.cc:769] Sending connection context
0806 11:36:52.396041 (+ 212us) client_negotiation.cc:240] Negotiation successful
0806 11:36:52.396223 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"mutex_wait_us":220,"thread_start_us":84,"threads_started":1}
I20260806 11:36:52.396889 30180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:52.390237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.193:40189 (local address 127.21.14.195:58269)
0806 11:36:52.390943 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:52.390960 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:52.391069 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:52.391444 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:52.391449 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:52.391475 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:36:52.391613 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:52.391623 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:52.392735 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:52.392738 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:52.394282 (+ 1544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:52.394299 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:52.395526 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:52.395530 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:52.395765 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:52.395835 (+ 70us) client_negotiation.cc:769] Sending connection context
0806 11:36:52.396041 (+ 206us) client_negotiation.cc:240] Negotiation successful
0806 11:36:52.396223 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"mutex_wait_us":572,"thread_start_us":84,"threads_started":1}
I20260806 11:36:52.396873 30098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:52.390423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:58269 (local address 127.21.14.193:40189)
0806 11:36:52.390555 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:52.390560 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:52.391035 (+ 475us) server_negotiation.cc:407] Connection header received
0806 11:36:52.391194 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:52.391197 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:52.391245 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:52.391297 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:36:52.391773 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:52.392518 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:52.394484 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:52.395207 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:52.395325 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:52.395383 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:52.396626 (+ 1243us) server_negotiation.cc:299] Negotiation successful
0806 11:36:52.396741 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 11:36:52.397521 29859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27d3535d45784e539e95fba5f0441b26" candidate_uuid: "f1d856374ce44857a183737b5e2b03ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92712864d28043d5aa2771fbc08fc004" is_pre_election: true
I20260806 11:36:52.397778 29710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27d3535d45784e539e95fba5f0441b26" candidate_uuid: "f1d856374ce44857a183737b5e2b03ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "741edc4087df42139936dace73d5f68a" is_pre_election: true
I20260806 11:36:52.397993 29859 raft_consensus.cc:2454] T 27d3535d45784e539e95fba5f0441b26 P 92712864d28043d5aa2771fbc08fc004 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1d856374ce44857a183737b5e2b03ba in term 0.
I20260806 11:36:52.398044 29710 raft_consensus.cc:2454] T 27d3535d45784e539e95fba5f0441b26 P 741edc4087df42139936dace73d5f68a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1d856374ce44857a183737b5e2b03ba in term 0.
I20260806 11:36:52.398777 29941 leader_election.cc:302] T 27d3535d45784e539e95fba5f0441b26 P f1d856374ce44857a183737b5e2b03ba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 741edc4087df42139936dace73d5f68a, 92712864d28043d5aa2771fbc08fc004, f1d856374ce44857a183737b5e2b03ba; no voters:
I20260806 11:36:52.399204 30107 raft_consensus.cc:2781] T 27d3535d45784e539e95fba5f0441b26 P f1d856374ce44857a183737b5e2b03ba [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 11:36:52.399286 30107 raft_consensus.cc:482] T 27d3535d45784e539e95fba5f0441b26 P f1d856374ce44857a183737b5e2b03ba [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 11:36:52.399328 30107 raft_consensus.cc:3037] T 27d3535d45784e539e95fba5f0441b26 P f1d856374ce44857a183737b5e2b03ba [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:36:52.400835 30107 raft_consensus.cc:504] T 27d3535d45784e539e95fba5f0441b26 P f1d856374ce44857a183737b5e2b03ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741edc4087df42139936dace73d5f68a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40189 } } peers { permanent_uuid: "92712864d28043d5aa2771fbc08fc004" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37297 } } peers { permanent_uuid: "f1d856374ce44857a183737b5e2b03ba" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38073 } }
I20260806 11:36:52.401372 30107 leader_election.cc:288] T 27d3535d45784e539e95fba5f0441b26 P f1d856374ce44857a183737b5e2b03ba [CANDIDATE]: Term 1 election: Requested vote from peers 741edc4087df42139936dace73d5f68a (127.21.14.193:40189), 92712864d28043d5aa2771fbc08fc004 (127.21.14.194:37297)
I20260806 11:36:52.401875 29859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27d3535d45784e539e95fba5f0441b26" candidate_uuid: "f1d856374ce44857a183737b5e2b03ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92712864d28043d5aa2771fbc08fc004"
I20260806 11:36:52.402072 29859 raft_consensus.cc:3037] T 27d3535d45784e539e95fba5f0441b26 P 92712864d28043d5aa2771fbc08fc004 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:36:52.402232 29710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27d3535d45784e539e95fba5f0441b26" candidate_uuid: "f1d856374ce44857a183737b5e2b03ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "741edc4087df42139936dace73d5f68a"
I20260806 11:36:52.402411 29710 raft_consensus.cc:3037] T 27d3535d45784e539e95fba5f0441b26 P 741edc4087df42139936dace73d5f68a [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:36:52.403445 29859 raft_consensus.cc:2454] T 27d3535d45784e539e95fba5f0441b26 P 92712864d28043d5aa2771fbc08fc004 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1d856374ce44857a183737b5e2b03ba in term 1.
I20260806 11:36:52.403801 29710 raft_consensus.cc:2454] T 27d3535d45784e539e95fba5f0441b26 P 741edc4087df42139936dace73d5f68a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1d856374ce44857a183737b5e2b03ba in term 1.
I20260806 11:36:52.403952 29941 leader_election.cc:302] T 27d3535d45784e539e95fba5f0441b26 P f1d856374ce44857a183737b5e2b03ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92712864d28043d5aa2771fbc08fc004, f1d856374ce44857a183737b5e2b03ba; no voters:
I20260806 11:36:52.404428 30107 raft_consensus.cc:2781] T 27d3535d45784e539e95fba5f0441b26 P f1d856374ce44857a183737b5e2b03ba [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:36:52.404687 30107 raft_consensus.cc:686] T 27d3535d45784e539e95fba5f0441b26 P f1d856374ce44857a183737b5e2b03ba [term 1 LEADER]: Becoming Leader. State: Replica: f1d856374ce44857a183737b5e2b03ba, State: Running, Role: LEADER
I20260806 11:36:52.404999 30107 consensus_queue.cc:237] T 27d3535d45784e539e95fba5f0441b26 P f1d856374ce44857a183737b5e2b03ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741edc4087df42139936dace73d5f68a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40189 } } peers { permanent_uuid: "92712864d28043d5aa2771fbc08fc004" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37297 } } peers { permanent_uuid: "f1d856374ce44857a183737b5e2b03ba" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38073 } }
I20260806 11:36:52.408412 29542 catalog_manager.cc:5495] T 27d3535d45784e539e95fba5f0441b26 P f1d856374ce44857a183737b5e2b03ba reported cstate change: term changed from 0 to 1, leader changed from <none> to f1d856374ce44857a183737b5e2b03ba (127.21.14.195). New cstate: current_term: 1 leader_uuid: "f1d856374ce44857a183737b5e2b03ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741edc4087df42139936dace73d5f68a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92712864d28043d5aa2771fbc08fc004" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37297 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f1d856374ce44857a183737b5e2b03ba" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38073 } health_report { overall_health: HEALTHY } } }
I20260806 11:36:52.426698 30099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:52.421769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46506 (local address 127.21.14.195:38073)
0806 11:36:52.421935 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:52.421940 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:52.421959 (+ 19us) server_negotiation.cc:407] Connection header received
0806 11:36:52.422116 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:52.422119 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:52.422206 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:52.422343 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 11:36:52.422660 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:52.423540 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:52.424597 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:52.424946 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:52.425099 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:52.425444 (+ 345us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 11:36:52.426152 (+ 708us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 11:36:52.426302 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:52.426458 (+ 156us) server_negotiation.cc:299] Negotiation successful
0806 11:36:52.426533 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 11:36:52.429970 29710 raft_consensus.cc:1261] T 27d3535d45784e539e95fba5f0441b26 P 741edc4087df42139936dace73d5f68a [term 1 FOLLOWER]: Refusing update from remote peer f1d856374ce44857a183737b5e2b03ba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 11:36:52.429970 29859 raft_consensus.cc:1261] T 27d3535d45784e539e95fba5f0441b26 P 92712864d28043d5aa2771fbc08fc004 [term 1 FOLLOWER]: Refusing update from remote peer f1d856374ce44857a183737b5e2b03ba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 11:36:52.430639 30181 consensus_queue.cc:1035] T 27d3535d45784e539e95fba5f0441b26 P f1d856374ce44857a183737b5e2b03ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "741edc4087df42139936dace73d5f68a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 11:36:52.430887 30107 consensus_queue.cc:1035] T 27d3535d45784e539e95fba5f0441b26 P f1d856374ce44857a183737b5e2b03ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "92712864d28043d5aa2771fbc08fc004" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 11:36:52.445124 29539 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60590:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 11:36:52.445326 29539 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60590:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 11:36:52.448117 29539 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ed204855a8dd4cd686fd550178b9563e: Sending DeleteTablet for 3 replicas of tablet 27d3535d45784e539e95fba5f0441b26
I20260806 11:36:52.448868 29690 tablet_service.cc:1514] Processing DeleteTablet for tablet 27d3535d45784e539e95fba5f0441b26 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:36:52 UTC) from {username='oryx', principal='oryx/127.21.14.254@KRBTEST.COM'} at 127.0.0.1:34180
I20260806 11:36:52.449107 30125 tablet_replica.cc:330] T 27d3535d45784e539e95fba5f0441b26 P 741edc4087df42139936dace73d5f68a: stopping tablet replica
I20260806 11:36:52.449244 30125 raft_consensus.cc:2229] T 27d3535d45784e539e95fba5f0441b26 P 741edc4087df42139936dace73d5f68a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 11:36:52.449416 30125 raft_consensus.cc:2258] T 27d3535d45784e539e95fba5f0441b26 P 741edc4087df42139936dace73d5f68a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 11:36:52.449519 29837 tablet_service.cc:1514] Processing DeleteTablet for tablet 27d3535d45784e539e95fba5f0441b26 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:36:52 UTC) from {username='oryx', principal='oryx/127.21.14.254@KRBTEST.COM'} at 127.0.0.1:60302
I20260806 11:36:52.449770 29995 tablet_service.cc:1514] Processing DeleteTablet for tablet 27d3535d45784e539e95fba5f0441b26 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:36:52 UTC) from {username='oryx', principal='oryx/127.21.14.254@KRBTEST.COM'} at 127.0.0.1:46500
I20260806 11:36:52.450021 30124 tablet_replica.cc:330] T 27d3535d45784e539e95fba5f0441b26 P 92712864d28043d5aa2771fbc08fc004: stopping tablet replica
I20260806 11:36:52.450201 30124 raft_consensus.cc:2229] T 27d3535d45784e539e95fba5f0441b26 P 92712864d28043d5aa2771fbc08fc004 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 11:36:52.450351 30124 raft_consensus.cc:2258] T 27d3535d45784e539e95fba5f0441b26 P 92712864d28043d5aa2771fbc08fc004 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 11:36:52.450541 30126 tablet_replica.cc:330] T 27d3535d45784e539e95fba5f0441b26 P f1d856374ce44857a183737b5e2b03ba: stopping tablet replica
I20260806 11:36:52.450554 30125 ts_tablet_manager.cc:1905] T 27d3535d45784e539e95fba5f0441b26 P 741edc4087df42139936dace73d5f68a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 11:36:52.450665 30126 raft_consensus.cc:2229] T 27d3535d45784e539e95fba5f0441b26 P f1d856374ce44857a183737b5e2b03ba [term 1 LEADER]: Raft consensus shutting down.
I20260806 11:36:52.451171 30126 raft_consensus.cc:2258] T 27d3535d45784e539e95fba5f0441b26 P f1d856374ce44857a183737b5e2b03ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 11:36:52.452414 29539 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60590:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 11:36:52.452926 29539 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 11:36:52.453567 30125 ts_tablet_manager.cc:1918] T 27d3535d45784e539e95fba5f0441b26 P 741edc4087df42139936dace73d5f68a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 11:36:52.453655 30125 log.cc:1196] T 27d3535d45784e539e95fba5f0441b26 P 741edc4087df42139936dace73d5f68a: Deleting WAL directory at /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/ts-0/wal/wals/27d3535d45784e539e95fba5f0441b26
I20260806 11:36:52.453938 30125 ts_tablet_manager.cc:1939] T 27d3535d45784e539e95fba5f0441b26 P 741edc4087df42139936dace73d5f68a: Deleting consensus metadata
I20260806 11:36:52.454896 30124 ts_tablet_manager.cc:1905] T 27d3535d45784e539e95fba5f0441b26 P 92712864d28043d5aa2771fbc08fc004: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 11:36:52.456463 30126 ts_tablet_manager.cc:1905] T 27d3535d45784e539e95fba5f0441b26 P f1d856374ce44857a183737b5e2b03ba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 11:36:52.457595 30124 ts_tablet_manager.cc:1918] T 27d3535d45784e539e95fba5f0441b26 P 92712864d28043d5aa2771fbc08fc004: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 11:36:52.457674 30124 log.cc:1196] T 27d3535d45784e539e95fba5f0441b26 P 92712864d28043d5aa2771fbc08fc004: Deleting WAL directory at /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/ts-1/wal/wals/27d3535d45784e539e95fba5f0441b26
I20260806 11:36:52.457947 30124 ts_tablet_manager.cc:1939] T 27d3535d45784e539e95fba5f0441b26 P 92712864d28043d5aa2771fbc08fc004: Deleting consensus metadata
I20260806 11:36:52.459000 29528 catalog_manager.cc:4841] TS 92712864d28043d5aa2771fbc08fc004 (127.21.14.194:37297): tablet 27d3535d45784e539e95fba5f0441b26 (table test-table [id=fec607f4a843415f9a58d9a5c2926cb3]) successfully deleted
I20260806 11:36:52.459731 30126 ts_tablet_manager.cc:1918] T 27d3535d45784e539e95fba5f0441b26 P f1d856374ce44857a183737b5e2b03ba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 11:36:52.459832 30126 log.cc:1196] T 27d3535d45784e539e95fba5f0441b26 P f1d856374ce44857a183737b5e2b03ba: Deleting WAL directory at /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/ts-2/wal/wals/27d3535d45784e539e95fba5f0441b26
I20260806 11:36:52.460108 30126 ts_tablet_manager.cc:1939] T 27d3535d45784e539e95fba5f0441b26 P f1d856374ce44857a183737b5e2b03ba: Deleting consensus metadata
I20260806 11:36:52.461004 29530 catalog_manager.cc:4841] TS f1d856374ce44857a183737b5e2b03ba (127.21.14.195:38073): tablet 27d3535d45784e539e95fba5f0441b26 (table test-table [id=fec607f4a843415f9a58d9a5c2926cb3]) successfully deleted
I20260806 11:36:52.464728 29690 tablet_service.cc:1467] Processing CreateTablet for tablet 3b00faced58248ff9d59d2fde6c8cc32 (DEFAULT_TABLE table=test-table [id=0eda62faf8684a519eb75eff1088ccf0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:36:52.464728 29837 tablet_service.cc:1467] Processing CreateTablet for tablet 3b00faced58248ff9d59d2fde6c8cc32 (DEFAULT_TABLE table=test-table [id=0eda62faf8684a519eb75eff1088ccf0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:36:52.465091 29837 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b00faced58248ff9d59d2fde6c8cc32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:52.465096 29690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b00faced58248ff9d59d2fde6c8cc32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:52.467341 29527 catalog_manager.cc:4841] TS 741edc4087df42139936dace73d5f68a (127.21.14.193:40189): tablet 27d3535d45784e539e95fba5f0441b26 (table test-table [id=fec607f4a843415f9a58d9a5c2926cb3]) successfully deleted
I20260806 11:36:52.467835 30101 tablet_bootstrap.cc:492] T 3b00faced58248ff9d59d2fde6c8cc32 P 741edc4087df42139936dace73d5f68a: Bootstrap starting.
I20260806 11:36:52.468071 29995 tablet_service.cc:1467] Processing CreateTablet for tablet 3b00faced58248ff9d59d2fde6c8cc32 (DEFAULT_TABLE table=test-table [id=0eda62faf8684a519eb75eff1088ccf0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:36:52.468403 29995 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b00faced58248ff9d59d2fde6c8cc32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:52.468784 30101 tablet_bootstrap.cc:654] T 3b00faced58248ff9d59d2fde6c8cc32 P 741edc4087df42139936dace73d5f68a: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:52.469226 30100 tablet_bootstrap.cc:492] T 3b00faced58248ff9d59d2fde6c8cc32 P 92712864d28043d5aa2771fbc08fc004: Bootstrap starting.
I20260806 11:36:52.470357 30100 tablet_bootstrap.cc:654] T 3b00faced58248ff9d59d2fde6c8cc32 P 92712864d28043d5aa2771fbc08fc004: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:52.470898 30101 tablet_bootstrap.cc:492] T 3b00faced58248ff9d59d2fde6c8cc32 P 741edc4087df42139936dace73d5f68a: No bootstrap required, opened a new log
I20260806 11:36:52.471012 30101 ts_tablet_manager.cc:1397] T 3b00faced58248ff9d59d2fde6c8cc32 P 741edc4087df42139936dace73d5f68a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 11:36:52.471481 30102 tablet_bootstrap.cc:492] T 3b00faced58248ff9d59d2fde6c8cc32 P f1d856374ce44857a183737b5e2b03ba: Bootstrap starting.
I20260806 11:36:52.471514 30101 raft_consensus.cc:348] T 3b00faced58248ff9d59d2fde6c8cc32 P 741edc4087df42139936dace73d5f68a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741edc4087df42139936dace73d5f68a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40189 } } peers { permanent_uuid: "92712864d28043d5aa2771fbc08fc004" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37297 } } peers { permanent_uuid: "f1d856374ce44857a183737b5e2b03ba" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38073 } }
I20260806 11:36:52.471639 30101 raft_consensus.cc:374] T 3b00faced58248ff9d59d2fde6c8cc32 P 741edc4087df42139936dace73d5f68a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:52.471669 30101 raft_consensus.cc:729] T 3b00faced58248ff9d59d2fde6c8cc32 P 741edc4087df42139936dace73d5f68a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 741edc4087df42139936dace73d5f68a, State: Initialized, Role: FOLLOWER
I20260806 11:36:52.471796 30101 consensus_queue.cc:260] T 3b00faced58248ff9d59d2fde6c8cc32 P 741edc4087df42139936dace73d5f68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741edc4087df42139936dace73d5f68a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40189 } } peers { permanent_uuid: "92712864d28043d5aa2771fbc08fc004" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37297 } } peers { permanent_uuid: "f1d856374ce44857a183737b5e2b03ba" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38073 } }
I20260806 11:36:52.472080 30101 ts_tablet_manager.cc:1428] T 3b00faced58248ff9d59d2fde6c8cc32 P 741edc4087df42139936dace73d5f68a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 11:36:52.472571 30102 tablet_bootstrap.cc:654] T 3b00faced58248ff9d59d2fde6c8cc32 P f1d856374ce44857a183737b5e2b03ba: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:52.474191 30100 tablet_bootstrap.cc:492] T 3b00faced58248ff9d59d2fde6c8cc32 P 92712864d28043d5aa2771fbc08fc004: No bootstrap required, opened a new log
I20260806 11:36:52.474324 30100 ts_tablet_manager.cc:1397] T 3b00faced58248ff9d59d2fde6c8cc32 P 92712864d28043d5aa2771fbc08fc004: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.001s
I20260806 11:36:52.474423 30102 tablet_bootstrap.cc:492] T 3b00faced58248ff9d59d2fde6c8cc32 P f1d856374ce44857a183737b5e2b03ba: No bootstrap required, opened a new log
I20260806 11:36:52.474511 30102 ts_tablet_manager.cc:1397] T 3b00faced58248ff9d59d2fde6c8cc32 P f1d856374ce44857a183737b5e2b03ba: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 11:36:52.474908 30100 raft_consensus.cc:348] T 3b00faced58248ff9d59d2fde6c8cc32 P 92712864d28043d5aa2771fbc08fc004 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741edc4087df42139936dace73d5f68a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40189 } } peers { permanent_uuid: "92712864d28043d5aa2771fbc08fc004" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37297 } } peers { permanent_uuid: "f1d856374ce44857a183737b5e2b03ba" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38073 } }
I20260806 11:36:52.475359 30100 raft_consensus.cc:374] T 3b00faced58248ff9d59d2fde6c8cc32 P 92712864d28043d5aa2771fbc08fc004 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:52.475404 30100 raft_consensus.cc:729] T 3b00faced58248ff9d59d2fde6c8cc32 P 92712864d28043d5aa2771fbc08fc004 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92712864d28043d5aa2771fbc08fc004, State: Initialized, Role: FOLLOWER
I20260806 11:36:52.475337 30102 raft_consensus.cc:348] T 3b00faced58248ff9d59d2fde6c8cc32 P f1d856374ce44857a183737b5e2b03ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741edc4087df42139936dace73d5f68a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40189 } } peers { permanent_uuid: "92712864d28043d5aa2771fbc08fc004" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37297 } } peers { permanent_uuid: "f1d856374ce44857a183737b5e2b03ba" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38073 } }
I20260806 11:36:52.475472 30102 raft_consensus.cc:374] T 3b00faced58248ff9d59d2fde6c8cc32 P f1d856374ce44857a183737b5e2b03ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:52.475508 30102 raft_consensus.cc:729] T 3b00faced58248ff9d59d2fde6c8cc32 P f1d856374ce44857a183737b5e2b03ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1d856374ce44857a183737b5e2b03ba, State: Initialized, Role: FOLLOWER
I20260806 11:36:52.475564 30100 consensus_queue.cc:260] T 3b00faced58248ff9d59d2fde6c8cc32 P 92712864d28043d5aa2771fbc08fc004 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741edc4087df42139936dace73d5f68a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40189 } } peers { permanent_uuid: "92712864d28043d5aa2771fbc08fc004" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37297 } } peers { permanent_uuid: "f1d856374ce44857a183737b5e2b03ba" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38073 } }
I20260806 11:36:52.475636 30102 consensus_queue.cc:260] T 3b00faced58248ff9d59d2fde6c8cc32 P f1d856374ce44857a183737b5e2b03ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741edc4087df42139936dace73d5f68a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40189 } } peers { permanent_uuid: "92712864d28043d5aa2771fbc08fc004" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37297 } } peers { permanent_uuid: "f1d856374ce44857a183737b5e2b03ba" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38073 } }
I20260806 11:36:52.475921 30100 ts_tablet_manager.cc:1428] T 3b00faced58248ff9d59d2fde6c8cc32 P 92712864d28043d5aa2771fbc08fc004: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260806 11:36:52.475927 30102 ts_tablet_manager.cc:1428] T 3b00faced58248ff9d59d2fde6c8cc32 P f1d856374ce44857a183737b5e2b03ba: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 11:36:52.500568 30119 raft_consensus.cc:482] T 3b00faced58248ff9d59d2fde6c8cc32 P 741edc4087df42139936dace73d5f68a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 11:36:52.500808 30119 raft_consensus.cc:504] T 3b00faced58248ff9d59d2fde6c8cc32 P 741edc4087df42139936dace73d5f68a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741edc4087df42139936dace73d5f68a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40189 } } peers { permanent_uuid: "92712864d28043d5aa2771fbc08fc004" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37297 } } peers { permanent_uuid: "f1d856374ce44857a183737b5e2b03ba" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38073 } }
I20260806 11:36:52.501456 30119 leader_election.cc:288] T 3b00faced58248ff9d59d2fde6c8cc32 P 741edc4087df42139936dace73d5f68a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 92712864d28043d5aa2771fbc08fc004 (127.21.14.194:37297), f1d856374ce44857a183737b5e2b03ba (127.21.14.195:38073)
I20260806 11:36:52.501909 29859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b00faced58248ff9d59d2fde6c8cc32" candidate_uuid: "741edc4087df42139936dace73d5f68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92712864d28043d5aa2771fbc08fc004" is_pre_election: true
I20260806 11:36:52.501978 30018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b00faced58248ff9d59d2fde6c8cc32" candidate_uuid: "741edc4087df42139936dace73d5f68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1d856374ce44857a183737b5e2b03ba" is_pre_election: true
I20260806 11:36:52.502095 29859 raft_consensus.cc:2454] T 3b00faced58248ff9d59d2fde6c8cc32 P 92712864d28043d5aa2771fbc08fc004 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 741edc4087df42139936dace73d5f68a in term 0.
I20260806 11:36:52.502148 30018 raft_consensus.cc:2454] T 3b00faced58248ff9d59d2fde6c8cc32 P f1d856374ce44857a183737b5e2b03ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 741edc4087df42139936dace73d5f68a in term 0.
I20260806 11:36:52.502454 29638 leader_election.cc:302] T 3b00faced58248ff9d59d2fde6c8cc32 P 741edc4087df42139936dace73d5f68a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 741edc4087df42139936dace73d5f68a, 92712864d28043d5aa2771fbc08fc004; no voters:
I20260806 11:36:52.502667 30119 raft_consensus.cc:2781] T 3b00faced58248ff9d59d2fde6c8cc32 P 741edc4087df42139936dace73d5f68a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 11:36:52.502738 30119 raft_consensus.cc:482] T 3b00faced58248ff9d59d2fde6c8cc32 P 741edc4087df42139936dace73d5f68a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 11:36:52.502777 30119 raft_consensus.cc:3037] T 3b00faced58248ff9d59d2fde6c8cc32 P 741edc4087df42139936dace73d5f68a [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:36:52.504020 30119 raft_consensus.cc:504] T 3b00faced58248ff9d59d2fde6c8cc32 P 741edc4087df42139936dace73d5f68a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741edc4087df42139936dace73d5f68a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40189 } } peers { permanent_uuid: "92712864d28043d5aa2771fbc08fc004" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37297 } } peers { permanent_uuid: "f1d856374ce44857a183737b5e2b03ba" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38073 } }
I20260806 11:36:52.504449 30119 leader_election.cc:288] T 3b00faced58248ff9d59d2fde6c8cc32 P 741edc4087df42139936dace73d5f68a [CANDIDATE]: Term 1 election: Requested vote from peers 92712864d28043d5aa2771fbc08fc004 (127.21.14.194:37297), f1d856374ce44857a183737b5e2b03ba (127.21.14.195:38073)
I20260806 11:36:52.504927 29859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b00faced58248ff9d59d2fde6c8cc32" candidate_uuid: "741edc4087df42139936dace73d5f68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92712864d28043d5aa2771fbc08fc004"
I20260806 11:36:52.505080 29859 raft_consensus.cc:3037] T 3b00faced58248ff9d59d2fde6c8cc32 P 92712864d28043d5aa2771fbc08fc004 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:36:52.505373 30018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b00faced58248ff9d59d2fde6c8cc32" candidate_uuid: "741edc4087df42139936dace73d5f68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1d856374ce44857a183737b5e2b03ba"
I20260806 11:36:52.505502 30018 raft_consensus.cc:3037] T 3b00faced58248ff9d59d2fde6c8cc32 P f1d856374ce44857a183737b5e2b03ba [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:36:52.506417 29859 raft_consensus.cc:2454] T 3b00faced58248ff9d59d2fde6c8cc32 P 92712864d28043d5aa2771fbc08fc004 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 741edc4087df42139936dace73d5f68a in term 1.
I20260806 11:36:52.506791 30018 raft_consensus.cc:2454] T 3b00faced58248ff9d59d2fde6c8cc32 P f1d856374ce44857a183737b5e2b03ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 741edc4087df42139936dace73d5f68a in term 1.
I20260806 11:36:52.506904 29638 leader_election.cc:302] T 3b00faced58248ff9d59d2fde6c8cc32 P 741edc4087df42139936dace73d5f68a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 741edc4087df42139936dace73d5f68a, 92712864d28043d5aa2771fbc08fc004; no voters:
I20260806 11:36:52.507205 30119 raft_consensus.cc:2781] T 3b00faced58248ff9d59d2fde6c8cc32 P 741edc4087df42139936dace73d5f68a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:36:52.507333 30119 raft_consensus.cc:686] T 3b00faced58248ff9d59d2fde6c8cc32 P 741edc4087df42139936dace73d5f68a [term 1 LEADER]: Becoming Leader. State: Replica: 741edc4087df42139936dace73d5f68a, State: Running, Role: LEADER
I20260806 11:36:52.507500 30119 consensus_queue.cc:237] T 3b00faced58248ff9d59d2fde6c8cc32 P 741edc4087df42139936dace73d5f68a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741edc4087df42139936dace73d5f68a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40189 } } peers { permanent_uuid: "92712864d28043d5aa2771fbc08fc004" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37297 } } peers { permanent_uuid: "f1d856374ce44857a183737b5e2b03ba" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38073 } }
I20260806 11:36:52.509621 29539 catalog_manager.cc:5495] T 3b00faced58248ff9d59d2fde6c8cc32 P 741edc4087df42139936dace73d5f68a reported cstate change: term changed from 0 to 1, leader changed from <none> to 741edc4087df42139936dace73d5f68a (127.21.14.193). New cstate: current_term: 1 leader_uuid: "741edc4087df42139936dace73d5f68a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "741edc4087df42139936dace73d5f68a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92712864d28043d5aa2771fbc08fc004" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37297 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f1d856374ce44857a183737b5e2b03ba" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38073 } health_report { overall_health: UNKNOWN } } }
I20260806 11:36:52.537057 30098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:52.533430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34214 (local address 127.21.14.193:40189)
0806 11:36:52.533559 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:52.533564 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:52.533579 (+ 15us) server_negotiation.cc:407] Connection header received
0806 11:36:52.533726 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:52.533729 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:52.533783 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:52.533866 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 11:36:52.534118 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:52.534933 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:52.535762 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:52.536110 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:52.536218 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:52.536597 (+ 379us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 11:36:52.536687 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 11:36:52.536761 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:52.536850 (+ 89us) server_negotiation.cc:299] Negotiation successful
0806 11:36:52.536905 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 11:36:52.547144 30098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:52.543908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:53467 (local address 127.21.14.193:40189)
0806 11:36:52.544034 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:52.544039 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:52.544053 (+ 14us) server_negotiation.cc:407] Connection header received
0806 11:36:52.544211 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:52.544214 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:52.544260 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:52.544341 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 11:36:52.544601 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:52.545108 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:52.546023 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:52.546240 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:52.546351 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:52.546595 (+ 244us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 11:36:52.546676 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 11:36:52.546776 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:52.546891 (+ 115us) server_negotiation.cc:299] Negotiation successful
0806 11:36:52.546960 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":7,"server-negotiator.queue_time_us":51}
I20260806 11:36:52.547144 30159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:52.543775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.193:40189 (local address 127.21.14.193:53467)
0806 11:36:52.543967 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:52.543985 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:52.544084 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:52.544348 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:52.544351 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:52.544357 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 11:36:52.544495 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:52.544507 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:52.545220 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:52.545225 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:52.545874 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:52.545883 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:52.546354 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:52.546357 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:52.546459 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:52.546482 (+ 23us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 11:36:52.546783 (+ 301us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 11:36:52.546791 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 11:36:52.546880 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 11:36:52.547004 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61,"mutex_wait_us":48}
I20260806 11:36:52.554527 30018 raft_consensus.cc:1261] T 3b00faced58248ff9d59d2fde6c8cc32 P f1d856374ce44857a183737b5e2b03ba [term 1 FOLLOWER]: Refusing update from remote peer 741edc4087df42139936dace73d5f68a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 11:36:52.554550 29859 raft_consensus.cc:1261] T 3b00faced58248ff9d59d2fde6c8cc32 P 92712864d28043d5aa2771fbc08fc004 [term 1 FOLLOWER]: Refusing update from remote peer 741edc4087df42139936dace73d5f68a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 11:36:52.555048 30108 consensus_queue.cc:1035] T 3b00faced58248ff9d59d2fde6c8cc32 P 741edc4087df42139936dace73d5f68a [LEADER]: Connected to new peer: Peer: permanent_uuid: "92712864d28043d5aa2771fbc08fc004" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 11:36:52.555253 30119 consensus_queue.cc:1035] T 3b00faced58248ff9d59d2fde6c8cc32 P 741edc4087df42139936dace73d5f68a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1d856374ce44857a183737b5e2b03ba" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38073 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 11:36:52.600248 29539 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60590:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 11:36:52.600466 29539 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60590:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 11:36:52.603397 29539 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ed204855a8dd4cd686fd550178b9563e: Sending DeleteTablet for 3 replicas of tablet 3b00faced58248ff9d59d2fde6c8cc32
I20260806 11:36:52.604457 29689 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b00faced58248ff9d59d2fde6c8cc32 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:36:52 UTC) from {username='oryx', principal='oryx/127.21.14.254@KRBTEST.COM'} at 127.0.0.1:34180
I20260806 11:36:52.604499 29837 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b00faced58248ff9d59d2fde6c8cc32 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:36:52 UTC) from {username='oryx', principal='oryx/127.21.14.254@KRBTEST.COM'} at 127.0.0.1:60302
I20260806 11:36:52.604688 30125 tablet_replica.cc:330] T 3b00faced58248ff9d59d2fde6c8cc32 P 741edc4087df42139936dace73d5f68a: stopping tablet replica
I20260806 11:36:52.604727 30124 tablet_replica.cc:330] T 3b00faced58248ff9d59d2fde6c8cc32 P 92712864d28043d5aa2771fbc08fc004: stopping tablet replica
I20260806 11:36:52.604876 30125 raft_consensus.cc:2229] T 3b00faced58248ff9d59d2fde6c8cc32 P 741edc4087df42139936dace73d5f68a [term 1 LEADER]: Raft consensus shutting down.
I20260806 11:36:52.604933 30124 raft_consensus.cc:2229] T 3b00faced58248ff9d59d2fde6c8cc32 P 92712864d28043d5aa2771fbc08fc004 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 11:36:52.605091 30124 raft_consensus.cc:2258] T 3b00faced58248ff9d59d2fde6c8cc32 P 92712864d28043d5aa2771fbc08fc004 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 11:36:52.605189 30125 raft_consensus.cc:2258] T 3b00faced58248ff9d59d2fde6c8cc32 P 741edc4087df42139936dace73d5f68a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 11:36:52.605830 29995 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b00faced58248ff9d59d2fde6c8cc32 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:36:52 UTC) from {username='oryx', principal='oryx/127.21.14.254@KRBTEST.COM'} at 127.0.0.1:46500
I20260806 11:36:52.606053 30126 tablet_replica.cc:330] T 3b00faced58248ff9d59d2fde6c8cc32 P f1d856374ce44857a183737b5e2b03ba: stopping tablet replica
I20260806 11:36:52.606177 30126 raft_consensus.cc:2229] T 3b00faced58248ff9d59d2fde6c8cc32 P f1d856374ce44857a183737b5e2b03ba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 11:36:52.606326 30126 raft_consensus.cc:2258] T 3b00faced58248ff9d59d2fde6c8cc32 P f1d856374ce44857a183737b5e2b03ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 11:36:52.606498 30125 ts_tablet_manager.cc:1905] T 3b00faced58248ff9d59d2fde6c8cc32 P 741edc4087df42139936dace73d5f68a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 11:36:52.606689 30124 ts_tablet_manager.cc:1905] T 3b00faced58248ff9d59d2fde6c8cc32 P 92712864d28043d5aa2771fbc08fc004: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 11:36:52.607612 30126 ts_tablet_manager.cc:1905] T 3b00faced58248ff9d59d2fde6c8cc32 P f1d856374ce44857a183737b5e2b03ba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 11:36:52.610641 30125 ts_tablet_manager.cc:1918] T 3b00faced58248ff9d59d2fde6c8cc32 P 741edc4087df42139936dace73d5f68a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 11:36:52.610641 30126 ts_tablet_manager.cc:1918] T 3b00faced58248ff9d59d2fde6c8cc32 P f1d856374ce44857a183737b5e2b03ba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 11:36:52.610735 30125 log.cc:1196] T 3b00faced58248ff9d59d2fde6c8cc32 P 741edc4087df42139936dace73d5f68a: Deleting WAL directory at /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/ts-0/wal/wals/3b00faced58248ff9d59d2fde6c8cc32
I20260806 11:36:52.610735 30126 log.cc:1196] T 3b00faced58248ff9d59d2fde6c8cc32 P f1d856374ce44857a183737b5e2b03ba: Deleting WAL directory at /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/ts-2/wal/wals/3b00faced58248ff9d59d2fde6c8cc32
I20260806 11:36:52.611042 30125 ts_tablet_manager.cc:1939] T 3b00faced58248ff9d59d2fde6c8cc32 P 741edc4087df42139936dace73d5f68a: Deleting consensus metadata
I20260806 11:36:52.611528 30126 ts_tablet_manager.cc:1939] T 3b00faced58248ff9d59d2fde6c8cc32 P f1d856374ce44857a183737b5e2b03ba: Deleting consensus metadata
I20260806 11:36:52.613461 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 29620
I20260806 11:36:52.619525 29530 catalog_manager.cc:4841] TS f1d856374ce44857a183737b5e2b03ba (127.21.14.195:38073): tablet 3b00faced58248ff9d59d2fde6c8cc32 (table test-table [id=0eda62faf8684a519eb75eff1088ccf0]) successfully deleted
I20260806 11:36:52.619642 30124 ts_tablet_manager.cc:1918] T 3b00faced58248ff9d59d2fde6c8cc32 P 92712864d28043d5aa2771fbc08fc004: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 11:36:52.619760 30124 log.cc:1196] T 3b00faced58248ff9d59d2fde6c8cc32 P 92712864d28043d5aa2771fbc08fc004: Deleting WAL directory at /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786016169045816-21563-0/minicluster-data/ts-1/wal/wals/3b00faced58248ff9d59d2fde6c8cc32
I20260806 11:36:52.620105 30124 ts_tablet_manager.cc:1939] T 3b00faced58248ff9d59d2fde6c8cc32 P 92712864d28043d5aa2771fbc08fc004: Deleting consensus metadata
I20260806 11:36:52.621078 29528 catalog_manager.cc:4841] TS 92712864d28043d5aa2771fbc08fc004 (127.21.14.194:37297): tablet 3b00faced58248ff9d59d2fde6c8cc32 (table test-table [id=0eda62faf8684a519eb75eff1088ccf0]) successfully deleted
W20260806 11:36:52.629751 29527 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.14.193:40189 (error 108)
I20260806 11:36:52.630755 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 29768
I20260806 11:36:52.631371 30196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:52.630790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.193:40189 (local address 127.0.0.1:34218)
0806 11:36:52.631194 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:52.631267 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.193:40189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":117,"threads_started":1}
W20260806 11:36:52.631481 29527 catalog_manager.cc:4584] TS 741edc4087df42139936dace73d5f68a (127.21.14.193:40189): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3b00faced58248ff9d59d2fde6c8cc32: Network error: Client connection negotiation failed: client connection to 127.21.14.193:40189: connect: Connection refused (error 111)
I20260806 11:36:52.644424 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 29919
I20260806 11:36:52.654820 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 29511
2026-08-06T11:36:52Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3386 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786016169045816-21563-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 11:36:52 dist-test-slave-7c35 krb5kdc[30199](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 11:36:52 dist-test-slave-7c35 krb5kdc[30199](info): set up 2 sockets
Aug 06 11:36:52 dist-test-slave-7c35 krb5kdc[30199](info): commencing operation
krb5kdc: starting...
W20260806 11:36:53.754406 21563 mini_kdc.cc:121] Time spent starting KDC: real 1.073s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Aug 06 11:36:53 dist-test-slave-7c35 krb5kdc[30199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016213, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T11:36:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T11:36:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.14.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab.
Entry for principal kudu/127.21.14.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab.
I20260806 11:36:53.913492 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786016169045816-21563-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786016169045816-21563-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786016169045816-21563-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:39385
--webserver_interface=127.21.14.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab
--principal=kudu/127.21.14.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:46161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.14.254:39385
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:36:54.045135 30214 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:36:54.045491 30214 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:36:54.045538 30214 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:36:54.050034 30214 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 11:36:54.050153 30214 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:36:54.050173 30214 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:36:54.050191 30214 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 11:36:54.050207 30214 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 11:36:54.056392 30214 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:46161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786016169045816-21563-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786016169045816-21563-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.14.254:39385
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:39385
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.14.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30214
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786016169045816-21563-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:36:54.057770 30214 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:36:54.059098 30214 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:36:54.070222 30220 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:54.070212 30219 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:54.070310 30214 server_base.cc:1034] running on GCE node
W20260806 11:36:54.070217 30222 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:54.071496 30214 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:36:54.072929 30214 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:36:54.074200 30214 hybrid_clock.cc:648] HybridClock initialized: now 1786016214074180 us; error 45 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-06T11:36:54Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1425 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 11:36:54 dist-test-slave-7c35 krb5kdc[30228](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 11:36:54 dist-test-slave-7c35 krb5kdc[30228](info): set up 2 sockets
Aug 06 11:36:54 dist-test-slave-7c35 krb5kdc[30228](info): commencing operation
krb5kdc: starting...
W20260806 11:36:55.193643 21563 mini_kdc.cc:121] Time spent starting KDC: real 1.086s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Aug 06 11:36:55 dist-test-slave-7c35 krb5kdc[30228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016215, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T11:36:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T11:36:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.14.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab.
Entry for principal kudu/127.21.14.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab.
I20260806 11:36:55.347368 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:45895
--webserver_interface=127.21.14.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab
--principal=kudu/127.21.14.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:42925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.14.254:45895
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:36:55.476301 30243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:36:55.476713 30243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:36:55.476804 30243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:36:55.481392 30243 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 11:36:55.481534 30243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:36:55.481572 30243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:36:55.481621 30243 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 11:36:55.481680 30243 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 11:36:55.487978 30243 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:42925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.14.254:45895
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab
--principal=kudu/127.21.14.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:45895
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.14.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30243
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:36:55.489355 30243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:36:55.490595 30243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:36:55.500315 30251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:55.500315 30248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:55.500339 30249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:55.500720 30243 server_base.cc:1034] running on GCE node
I20260806 11:36:55.501516 30243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:36:55.502988 30243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:36:55.504227 30243 hybrid_clock.cc:648] HybridClock initialized: now 1786016215504205 us; error 46 us; skew 500 ppm
Aug 06 11:36:55 dist-test-slave-7c35 krb5kdc[30228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016215, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:36:55.507556 30243 init.cc:377] Logged in from keytab as kudu/127.21.14.254@KRBTEST.COM (short username kudu)
I20260806 11:36:55.508767 30243 webserver.cc:460] Webserver started at http://127.21.14.254:37335/ using document root <none> and password file <none>
I20260806 11:36:55.509445 30243 fs_manager.cc:359] Metadata directory not provided
I20260806 11:36:55.509553 30243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:36:55.509824 30243 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:36:55.512274 30243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/master-0/data/instance:
uuid: "5fc5cd921ca240ab8220d493662a7f88"
format_stamp: "Formatted at 2026-08-06 11:36:55 on dist-test-slave-7c35"
server_key: "d354f89992cc5c069441bd66e6772ce5"
server_key_iv: "6130412f7b43357fa66fa7662b7298ac"
server_key_version: "encryptionkey@0"
I20260806 11:36:55.512943 30243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/master-0/wal/instance:
uuid: "5fc5cd921ca240ab8220d493662a7f88"
format_stamp: "Formatted at 2026-08-06 11:36:55 on dist-test-slave-7c35"
server_key: "d354f89992cc5c069441bd66e6772ce5"
server_key_iv: "6130412f7b43357fa66fa7662b7298ac"
server_key_version: "encryptionkey@0"
I20260806 11:36:55.517400 30243 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 11:36:55.520493 30258 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:55.522174 30243 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 11:36:55.522495 30243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/master-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/master-0/wal
uuid: "5fc5cd921ca240ab8220d493662a7f88"
format_stamp: "Formatted at 2026-08-06 11:36:55 on dist-test-slave-7c35"
server_key: "d354f89992cc5c069441bd66e6772ce5"
server_key_iv: "6130412f7b43357fa66fa7662b7298ac"
server_key_version: "encryptionkey@0"
I20260806 11:36:55.522666 30243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:36:55.544368 30243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:36:55.548368 30243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:36:55.548694 30243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:36:55.558552 30243 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.254:45895
I20260806 11:36:55.558565 30318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.254:45895 every 8 connection(s)
I20260806 11:36:55.559975 30243 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
I20260806 11:36:55.563714 30319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:36:55.569346 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 30243
I20260806 11:36:55.569526 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/master-0/wal/instance
I20260806 11:36:55.569800 21563 external_mini_cluster.cc:1428] Setting key f97ed2b3b8e6762cbe6b974ccc5d06cf
I20260806 11:36:55.570693 30319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fc5cd921ca240ab8220d493662a7f88: Bootstrap starting.
I20260806 11:36:55.573643 30319 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5fc5cd921ca240ab8220d493662a7f88: Neither blocks nor log segments found. Creating new log.
I20260806 11:36:55.574626 30319 log.cc:824] T 00000000000000000000000000000000 P 5fc5cd921ca240ab8220d493662a7f88: Log is configured to *not* fsync() on all Append() calls
I20260806 11:36:55.577582 30319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fc5cd921ca240ab8220d493662a7f88: No bootstrap required, opened a new log
Aug 06 11:36:55 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016215, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:36:55.582474 30319 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5fc5cd921ca240ab8220d493662a7f88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fc5cd921ca240ab8220d493662a7f88" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 45895 } }
I20260806 11:36:55.582898 30319 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5fc5cd921ca240ab8220d493662a7f88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:36:55.582983 30319 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5fc5cd921ca240ab8220d493662a7f88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fc5cd921ca240ab8220d493662a7f88, State: Initialized, Role: FOLLOWER
I20260806 11:36:55.583662 30319 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5fc5cd921ca240ab8220d493662a7f88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fc5cd921ca240ab8220d493662a7f88" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 45895 } }
I20260806 11:36:55.583863 30319 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5fc5cd921ca240ab8220d493662a7f88 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 11:36:55.583940 30319 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5fc5cd921ca240ab8220d493662a7f88 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 11:36:55.584059 30319 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5fc5cd921ca240ab8220d493662a7f88 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:36:55.585321 30319 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5fc5cd921ca240ab8220d493662a7f88 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fc5cd921ca240ab8220d493662a7f88" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 45895 } }
I20260806 11:36:55.585786 30319 leader_election.cc:302] T 00000000000000000000000000000000 P 5fc5cd921ca240ab8220d493662a7f88 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5fc5cd921ca240ab8220d493662a7f88; no voters:
I20260806 11:36:55.586251 30319 leader_election.cc:288] T 00000000000000000000000000000000 P 5fc5cd921ca240ab8220d493662a7f88 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 11:36:55.586431 30324 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5fc5cd921ca240ab8220d493662a7f88 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:36:55.586743 30324 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5fc5cd921ca240ab8220d493662a7f88 [term 1 LEADER]: Becoming Leader. State: Replica: 5fc5cd921ca240ab8220d493662a7f88, State: Running, Role: LEADER
I20260806 11:36:55.587142 30324 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5fc5cd921ca240ab8220d493662a7f88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fc5cd921ca240ab8220d493662a7f88" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 45895 } }
I20260806 11:36:55.587888 30319 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5fc5cd921ca240ab8220d493662a7f88 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 11:36:55.589226 30322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:55.571547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50684 (local address 127.21.14.254:45895)
0806 11:36:55.572170 (+ 623us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:55.572190 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:55.572238 (+ 48us) server_negotiation.cc:407] Connection header received
0806 11:36:55.573071 (+ 833us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:55.573094 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:55.573529 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:55.573975 (+ 446us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:55.575002 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:55.576469 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:55.577556 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:55.578322 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:55.578510 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:55.581757 (+ 3247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:55.581765 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:55.581789 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:55.581839 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:55.584332 (+ 2493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:55.585196 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:55.585205 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:55.585214 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:55.585414 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:55.585816 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:55.585819 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:55.585821 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:55.586467 (+ 646us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:55.586706 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:55.587666 (+ 960us) server_negotiation.cc:299] Negotiation successful
0806 11:36:55.588030 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":437,"thread_start_us":175,"threads_started":1}
I20260806 11:36:55.589390 30326 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fc5cd921ca240ab8220d493662a7f88 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5fc5cd921ca240ab8220d493662a7f88. Latest consensus state: current_term: 1 leader_uuid: "5fc5cd921ca240ab8220d493662a7f88" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fc5cd921ca240ab8220d493662a7f88" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 45895 } } }
I20260806 11:36:55.589548 30326 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fc5cd921ca240ab8220d493662a7f88 [sys.catalog]: This master's current role is: LEADER
I20260806 11:36:55.589677 30325 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fc5cd921ca240ab8220d493662a7f88 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5fc5cd921ca240ab8220d493662a7f88" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fc5cd921ca240ab8220d493662a7f88" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 45895 } } }
I20260806 11:36:55.589783 30325 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fc5cd921ca240ab8220d493662a7f88 [sys.catalog]: This master's current role is: LEADER
W20260806 11:36:55.593787 30339 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5fc5cd921ca240ab8220d493662a7f88: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 11:36:55.593924 30339 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 11:36:55.594050 30334 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 11:36:55.595199 30334 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 11:36:55.602945 30334 catalog_manager.cc:1347] Generated new cluster ID: 1f36bd5082aa4373ba20756619e62cfd
I20260806 11:36:55.603065 30334 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 11:36:55.676909 30334 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 11:36:55.678048 30334 catalog_manager.cc:1504] Loading token signing keys...
I20260806 11:36:55.690577 30334 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5fc5cd921ca240ab8220d493662a7f88: Generated new TSK 0
I20260806 11:36:55.691613 30334 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.14.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab.
Entry for principal kudu/127.21.14.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab.
I20260806 11:36:55.742660 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.193:0
--local_ip_for_outbound_sockets=127.21.14.193
--webserver_interface=127.21.14.193
--webserver_port=0
--tserver_master_addrs=127.21.14.254:45895
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab
--principal=kudu/127.21.14.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:42925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:36:55.867568 30346 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:36:55.867897 30346 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:36:55.867944 30346 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:36:55.872355 30346 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:36:55.872447 30346 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:36:55.872517 30346 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.193
I20260806 11:36:55.878500 30346 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:42925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab
--principal=kudu/127.21.14.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.14.193
--webserver_port=0
--tserver_master_addrs=127.21.14.254:45895
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30346
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.193
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:36:55.879845 30346 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:36:55.880978 30346 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:36:55.892243 30354 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:55.892238 30351 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:55.892238 30352 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:55.892601 30346 server_base.cc:1034] running on GCE node
I20260806 11:36:55.893128 30346 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:36:55.893878 30346 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:36:55.895081 30346 hybrid_clock.cc:648] HybridClock initialized: now 1786016215895037 us; error 57 us; skew 500 ppm
Aug 06 11:36:55 dist-test-slave-7c35 krb5kdc[30228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016215, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:36:55.898389 30346 init.cc:377] Logged in from keytab as kudu/127.21.14.193@KRBTEST.COM (short username kudu)
I20260806 11:36:55.899549 30346 webserver.cc:460] Webserver started at http://127.21.14.193:40067/ using document root <none> and password file <none>
I20260806 11:36:55.900211 30346 fs_manager.cc:359] Metadata directory not provided
I20260806 11:36:55.900316 30346 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:36:55.900586 30346 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:36:55.903033 30346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-0/data/instance:
uuid: "6a622d7a17da41079e47dd97342f467e"
format_stamp: "Formatted at 2026-08-06 11:36:55 on dist-test-slave-7c35"
server_key: "1f66265cc9689d1018fc58dfc2128411"
server_key_iv: "5c53fdddb3d1e35cb0ea517676e1d3ce"
server_key_version: "encryptionkey@0"
I20260806 11:36:55.903718 30346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-0/wal/instance:
uuid: "6a622d7a17da41079e47dd97342f467e"
format_stamp: "Formatted at 2026-08-06 11:36:55 on dist-test-slave-7c35"
server_key: "1f66265cc9689d1018fc58dfc2128411"
server_key_iv: "5c53fdddb3d1e35cb0ea517676e1d3ce"
server_key_version: "encryptionkey@0"
I20260806 11:36:55.908120 30346 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 11:36:55.911391 30361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:55.912940 30346 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 11:36:55.913133 30346 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-0/wal
uuid: "6a622d7a17da41079e47dd97342f467e"
format_stamp: "Formatted at 2026-08-06 11:36:55 on dist-test-slave-7c35"
server_key: "1f66265cc9689d1018fc58dfc2128411"
server_key_iv: "5c53fdddb3d1e35cb0ea517676e1d3ce"
server_key_version: "encryptionkey@0"
I20260806 11:36:55.913291 30346 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:36:55.952195 30346 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:36:55.955888 30346 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:36:55.956141 30346 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:36:55.956854 30346 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:36:55.957845 30346 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:36:55.957927 30346 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:55.958003 30346 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:36:55.958048 30346 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:55.968019 30346 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.193:44447
I20260806 11:36:55.968050 30483 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.193:44447 every 8 connection(s)
I20260806 11:36:55.969022 30346 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
I20260806 11:36:55.971854 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 30346
I20260806 11:36:55.971998 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-0/wal/instance
I20260806 11:36:55.972311 21563 external_mini_cluster.cc:1428] Setting key 354c0c76e342b73a32d672f5e838ae3b
Aug 06 11:36:55 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016215, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.193@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.14.194@KRBTEST.COM; defaulting to no policy
I20260806 11:36:55.985739 30322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:55.971147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:59877 (local address 127.21.14.254:45895)
0806 11:36:55.971306 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:55.971309 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:55.972525 (+ 1216us) server_negotiation.cc:407] Connection header received
0806 11:36:55.974144 (+ 1619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:55.974147 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:55.974205 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:55.974280 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:55.975683 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:55.976572 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:55.977894 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:55.978320 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:55.978456 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:55.981638 (+ 3182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:55.981641 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:55.981643 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:55.981674 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:55.983403 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:55.984030 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:55.984033 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:55.984034 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:55.984122 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:55.984554 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:55.984557 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:55.984558 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:55.984768 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:55.984849 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:55.985409 (+ 560us) server_negotiation.cc:299] Negotiation successful
0806 11:36:55.985542 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.194@KRBTEST.COM" created.
I20260806 11:36:55.986768 30486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:55.971666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:45895 (local address 127.21.14.193:59877)
0806 11:36:55.972295 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:55.972376 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:55.973879 (+ 1503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:55.974477 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:55.974484 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:55.974955 (+ 471us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:55.975499 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:55.975514 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:55.976742 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:55.976745 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:55.977748 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:55.977759 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:55.978546 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:55.978550 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:55.978738 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:55.979520 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:55.979539 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:55.981387 (+ 1848us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:55.983576 (+ 2189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:55.983583 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:55.983602 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:55.983910 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:55.984211 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:55.984213 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:55.984215 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:55.984429 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:55.984865 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:55.984871 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:55.985160 (+ 289us) client_negotiation.cc:769] Sending connection context
0806 11:36:55.985333 (+ 173us) client_negotiation.cc:240] Negotiation successful
0806 11:36:55.985644 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":174,"threads_started":1}
I20260806 11:36:55.988399 30484 heartbeater.cc:347] Connected to a master server at 127.21.14.254:45895
I20260806 11:36:55.988729 30484 heartbeater.cc:464] Registering TS with master...
I20260806 11:36:55.989408 30484 heartbeater.cc:511] Master 127.21.14.254:45895 requested a full tablet report, sending...
I20260806 11:36:55.991434 30277 ts_manager.cc:194] Registered new tserver with Master: 6a622d7a17da41079e47dd97342f467e (127.21.14.193:44447)
I20260806 11:36:55.993265 30277 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.14.193@KRBTEST.COM'} at 127.21.14.193:59877
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab.
Entry for principal kudu/127.21.14.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab.
I20260806 11:36:56.007738 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.194:0
--local_ip_for_outbound_sockets=127.21.14.194
--webserver_interface=127.21.14.194
--webserver_port=0
--tserver_master_addrs=127.21.14.254:45895
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab
--principal=kudu/127.21.14.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:42925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:36:56.154157 30489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:36:56.154510 30489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:36:56.154570 30489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:36:56.159283 30489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:36:56.159374 30489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:36:56.159442 30489 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.194
I20260806 11:36:56.165576 30489 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:42925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab
--principal=kudu/127.21.14.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.14.194
--webserver_port=0
--tserver_master_addrs=127.21.14.254:45895
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30489
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.194
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:36:56.166806 30489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:36:56.167984 30489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:36:56.180532 30497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:56.180574 30494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:56.180574 30495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:56.180891 30489 server_base.cc:1034] running on GCE node
I20260806 11:36:56.181566 30489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:36:56.182359 30489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:36:56.183619 30489 hybrid_clock.cc:648] HybridClock initialized: now 1786016216183601 us; error 34 us; skew 500 ppm
Aug 06 11:36:56 dist-test-slave-7c35 krb5kdc[30228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016216, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:36:56.187381 30489 init.cc:377] Logged in from keytab as kudu/127.21.14.194@KRBTEST.COM (short username kudu)
I20260806 11:36:56.188673 30489 webserver.cc:460] Webserver started at http://127.21.14.194:46533/ using document root <none> and password file <none>
I20260806 11:36:56.189486 30489 fs_manager.cc:359] Metadata directory not provided
I20260806 11:36:56.189577 30489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:36:56.189782 30489 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:36:56.192451 30489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-1/data/instance:
uuid: "cded16d898934ba09215b52922e630b2"
format_stamp: "Formatted at 2026-08-06 11:36:56 on dist-test-slave-7c35"
server_key: "0e3ef3810d9e44ebfb709b01f4c385ab"
server_key_iv: "3b1f8ea6cf91cc5fdbe27c1c61eeead7"
server_key_version: "encryptionkey@0"
I20260806 11:36:56.193138 30489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-1/wal/instance:
uuid: "cded16d898934ba09215b52922e630b2"
format_stamp: "Formatted at 2026-08-06 11:36:56 on dist-test-slave-7c35"
server_key: "0e3ef3810d9e44ebfb709b01f4c385ab"
server_key_iv: "3b1f8ea6cf91cc5fdbe27c1c61eeead7"
server_key_version: "encryptionkey@0"
I20260806 11:36:56.197890 30489 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 11:36:56.201182 30504 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:56.202979 30489 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 11:36:56.203146 30489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-1/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-1/wal
uuid: "cded16d898934ba09215b52922e630b2"
format_stamp: "Formatted at 2026-08-06 11:36:56 on dist-test-slave-7c35"
server_key: "0e3ef3810d9e44ebfb709b01f4c385ab"
server_key_iv: "3b1f8ea6cf91cc5fdbe27c1c61eeead7"
server_key_version: "encryptionkey@0"
I20260806 11:36:56.203292 30489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:36:56.228490 30489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:36:56.232364 30489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:36:56.232620 30489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:36:56.233314 30489 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:36:56.234277 30489 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:36:56.234336 30489 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:56.234381 30489 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:36:56.234397 30489 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:56.245842 30489 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.194:35237
I20260806 11:36:56.245860 30626 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.194:35237 every 8 connection(s)
I20260806 11:36:56.247195 30489 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
I20260806 11:36:56.247368 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 30489
I20260806 11:36:56.247480 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-1/wal/instance
I20260806 11:36:56.247848 21563 external_mini_cluster.cc:1428] Setting key 2414d9ab27b46ec1d15ab12bdee9af81
Aug 06 11:36:56 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016216, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.194@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.14.195@KRBTEST.COM; defaulting to no policy
I20260806 11:36:56.266471 30322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:56.249593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:40313 (local address 127.21.14.254:45895)
0806 11:36:56.249739 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:56.249744 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:56.250993 (+ 1249us) server_negotiation.cc:407] Connection header received
0806 11:36:56.252851 (+ 1858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:56.252855 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:56.252928 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:56.253012 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:56.254436 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:56.255362 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:56.256910 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:56.257333 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:56.257497 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:56.261269 (+ 3772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:56.261276 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:56.261281 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:56.261323 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:56.263348 (+ 2025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:56.264210 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:56.264215 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:56.264224 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:56.264329 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:56.264871 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:56.264875 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:56.264876 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:56.265125 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:56.265224 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:56.266098 (+ 874us) server_negotiation.cc:299] Negotiation successful
0806 11:36:56.266263 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.195@KRBTEST.COM" created.
I20260806 11:36:56.267539 30630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:56.250155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:45895 (local address 127.21.14.194:40313)
0806 11:36:56.250774 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:56.250879 (+ 105us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:56.252565 (+ 1686us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:56.253234 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:56.253243 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:56.253666 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:56.254241 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:56.254256 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:56.255628 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:56.255632 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:56.256726 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:56.256740 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:56.257596 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:56.257600 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:56.257814 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:56.258629 (+ 815us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:56.258655 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:56.260950 (+ 2295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:56.263605 (+ 2655us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:56.263613 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:56.263634 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:56.264036 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:56.264481 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:56.264484 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:56.264486 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:56.264710 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:56.265343 (+ 633us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:56.265355 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:56.265748 (+ 393us) client_negotiation.cc:769] Sending connection context
0806 11:36:56.265962 (+ 214us) client_negotiation.cc:240] Negotiation successful
0806 11:36:56.266336 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":144,"threads_started":1}
I20260806 11:36:56.269151 30627 heartbeater.cc:347] Connected to a master server at 127.21.14.254:45895
I20260806 11:36:56.269474 30627 heartbeater.cc:464] Registering TS with master...
I20260806 11:36:56.270177 30627 heartbeater.cc:511] Master 127.21.14.254:45895 requested a full tablet report, sending...
I20260806 11:36:56.271708 30277 ts_manager.cc:194] Registered new tserver with Master: cded16d898934ba09215b52922e630b2 (127.21.14.194:35237)
I20260806 11:36:56.272408 30277 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.14.194@KRBTEST.COM'} at 127.21.14.194:40313
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab.
Entry for principal kudu/127.21.14.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab.
I20260806 11:36:56.288187 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.195:0
--local_ip_for_outbound_sockets=127.21.14.195
--webserver_interface=127.21.14.195
--webserver_port=0
--tserver_master_addrs=127.21.14.254:45895
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab
--principal=kudu/127.21.14.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:42925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:36:56.415174 30633 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:36:56.415509 30633 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:36:56.415557 30633 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:36:56.420184 30633 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:36:56.420287 30633 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:36:56.420354 30633 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.195
I20260806 11:36:56.426508 30633 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:42925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab
--principal=kudu/127.21.14.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.14.195
--webserver_port=0
--tserver_master_addrs=127.21.14.254:45895
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30633
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.195
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:36:56.427906 30633 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:36:56.429088 30633 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 11:36:56.440434 30633 server_base.cc:1034] running on GCE node
W20260806 11:36:56.440373 30639 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:56.440373 30641 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:56.440371 30638 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:56.441130 30633 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:36:56.441900 30633 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:36:56.443120 30633 hybrid_clock.cc:648] HybridClock initialized: now 1786016216443077 us; error 59 us; skew 500 ppm
Aug 06 11:36:56 dist-test-slave-7c35 krb5kdc[30228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016216, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:36:56.446236 30633 init.cc:377] Logged in from keytab as kudu/127.21.14.195@KRBTEST.COM (short username kudu)
I20260806 11:36:56.447471 30633 webserver.cc:460] Webserver started at http://127.21.14.195:33377/ using document root <none> and password file <none>
I20260806 11:36:56.448135 30633 fs_manager.cc:359] Metadata directory not provided
I20260806 11:36:56.448247 30633 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:36:56.448503 30633 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:36:56.450989 30633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-2/data/instance:
uuid: "781b6728bbc34238b584293ee898c245"
format_stamp: "Formatted at 2026-08-06 11:36:56 on dist-test-slave-7c35"
server_key: "3b38719225212651d9234b55061db409"
server_key_iv: "90faa2167f89c8651667acbea6d4c318"
server_key_version: "encryptionkey@0"
I20260806 11:36:56.451679 30633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-2/wal/instance:
uuid: "781b6728bbc34238b584293ee898c245"
format_stamp: "Formatted at 2026-08-06 11:36:56 on dist-test-slave-7c35"
server_key: "3b38719225212651d9234b55061db409"
server_key_iv: "90faa2167f89c8651667acbea6d4c318"
server_key_version: "encryptionkey@0"
I20260806 11:36:56.456158 30633 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 11:36:56.459197 30649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:56.460661 30633 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 11:36:56.460835 30633 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-2/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-2/wal
uuid: "781b6728bbc34238b584293ee898c245"
format_stamp: "Formatted at 2026-08-06 11:36:56 on dist-test-slave-7c35"
server_key: "3b38719225212651d9234b55061db409"
server_key_iv: "90faa2167f89c8651667acbea6d4c318"
server_key_version: "encryptionkey@0"
I20260806 11:36:56.460978 30633 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:36:56.482455 30633 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:36:56.486953 30633 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:36:56.487214 30633 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:36:56.487926 30633 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:36:56.489058 30633 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:36:56.489131 30633 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:56.489214 30633 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:36:56.489269 30633 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:56.500170 30633 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.195:43735
I20260806 11:36:56.500204 30774 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.195:43735 every 8 connection(s)
I20260806 11:36:56.501253 30633 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
I20260806 11:36:56.507992 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 30633
I20260806 11:36:56.508175 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-2/wal/instance
I20260806 11:36:56.508584 21563 external_mini_cluster.cc:1428] Setting key 11125bb80f0b0c7bf309617f2c379e23
Aug 06 11:36:56 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016216, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.195@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:36:56.517762 30322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:56.503480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:55811 (local address 127.21.14.254:45895)
0806 11:36:56.503655 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:56.503658 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:56.504640 (+ 982us) server_negotiation.cc:407] Connection header received
0806 11:36:56.506464 (+ 1824us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:56.506468 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:56.506542 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:56.506642 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:56.508294 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:56.509074 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:56.510648 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:56.510993 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:56.511129 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:56.513876 (+ 2747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:56.513880 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:56.513883 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:56.513937 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:56.515369 (+ 1432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:56.515961 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:56.515965 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:56.515967 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:56.516059 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:56.516429 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:56.516436 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:56.516440 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:56.516652 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:56.516768 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:56.517426 (+ 658us) server_negotiation.cc:299] Negotiation successful
0806 11:36:56.517582 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 11:36:56.519011 30778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:56.503864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:45895 (local address 127.21.14.195:55811)
0806 11:36:56.504445 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:56.504497 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:56.506042 (+ 1545us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:56.506903 (+ 861us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:56.506920 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:56.507468 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:56.508069 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:56.508086 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:56.509260 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:56.509264 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:56.510495 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:56.510508 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:56.511183 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:56.511187 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:56.511381 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:56.512001 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:56.512020 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:56.513673 (+ 1653us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:36:56.515511 (+ 1838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:56.515518 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:56.515541 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:56.515839 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:56.516159 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:36:56.516163 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:36:56.516165 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:36:56.516328 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:36:56.516768 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:36:56.516777 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:36:56.517074 (+ 297us) client_negotiation.cc:769] Sending connection context
0806 11:36:56.517383 (+ 309us) client_negotiation.cc:240] Negotiation successful
0806 11:36:56.517797 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":165,"threads_started":1}
I20260806 11:36:56.520373 30775 heartbeater.cc:347] Connected to a master server at 127.21.14.254:45895
I20260806 11:36:56.520653 30775 heartbeater.cc:464] Registering TS with master...
I20260806 11:36:56.521186 30775 heartbeater.cc:511] Master 127.21.14.254:45895 requested a full tablet report, sending...
I20260806 11:36:56.522588 30277 ts_manager.cc:194] Registered new tserver with Master: 781b6728bbc34238b584293ee898c245 (127.21.14.195:43735)
I20260806 11:36:56.523237 21563 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 11:36:56.523445 30277 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.14.195@KRBTEST.COM'} at 127.21.14.195:55811
WARNING: no policy specified for oryx/127.21.14.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.14.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.14.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/krb5kdc/oryx_127.21.14.193.keytab.
Entry for principal oryx/127.21.14.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/krb5kdc/oryx_127.21.14.193.keytab.
WARNING: no policy specified for oryx/127.21.14.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.14.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.14.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/krb5kdc/oryx_127.21.14.194.keytab.
Entry for principal oryx/127.21.14.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/krb5kdc/oryx_127.21.14.194.keytab.
WARNING: no policy specified for oryx/127.21.14.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.14.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.14.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/krb5kdc/oryx_127.21.14.195.keytab.
Entry for principal oryx/127.21.14.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/krb5kdc/oryx_127.21.14.195.keytab.
I20260806 11:36:56.627614 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 30346
I20260806 11:36:56.636657 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 30489
I20260806 11:36:56.645257 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 30633
I20260806 11:36:56.655267 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 30243
I20260806 11:36:56.664348 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:45895
--webserver_interface=127.21.14.254
--webserver_port=37335
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab
--principal=kudu/127.21.14.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:42925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.14.254:45895
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:36:56.786672 30785 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:36:56.787115 30785 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:36:56.787206 30785 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:36:56.791631 30785 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 11:36:56.791759 30785 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:36:56.791797 30785 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:36:56.791841 30785 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 11:36:56.791877 30785 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 11:36:56.797832 30785 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:42925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.14.254:45895
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab
--principal=kudu/127.21.14.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:45895
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.14.254
--webserver_port=37335
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30785
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:36:56.799202 30785 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:36:56.800414 30785 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:36:56.810485 30790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:56.810474 30791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:56.810536 30785 server_base.cc:1034] running on GCE node
W20260806 11:36:56.810439 30793 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:56.811561 30785 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:36:56.812875 30785 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:36:56.814096 30785 hybrid_clock.cc:648] HybridClock initialized: now 1786016216814076 us; error 41 us; skew 500 ppm
Aug 06 11:36:56 dist-test-slave-7c35 krb5kdc[30228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016216, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:36:56.817364 30785 init.cc:377] Logged in from keytab as kudu/127.21.14.254@KRBTEST.COM (short username kudu)
I20260806 11:36:56.818542 30785 webserver.cc:460] Webserver started at http://127.21.14.254:37335/ using document root <none> and password file <none>
I20260806 11:36:56.819307 30785 fs_manager.cc:359] Metadata directory not provided
I20260806 11:36:56.819446 30785 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:36:56.823904 30785 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 11:36:56.826691 30801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:56.828200 30785 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 11:36:56.828388 30785 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/master-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/master-0/wal
uuid: "5fc5cd921ca240ab8220d493662a7f88"
format_stamp: "Formatted at 2026-08-06 11:36:55 on dist-test-slave-7c35"
server_key: "d354f89992cc5c069441bd66e6772ce5"
server_key_iv: "6130412f7b43357fa66fa7662b7298ac"
server_key_version: "encryptionkey@0"
I20260806 11:36:56.829053 30785 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:36:56.854651 30785 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:36:56.858462 30785 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:36:56.858714 30785 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:36:56.867129 30785 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.254:45895
I20260806 11:36:56.867146 30859 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.254:45895 every 8 connection(s)
I20260806 11:36:56.868419 30785 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
I20260806 11:36:56.873090 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 30785
I20260806 11:36:56.873371 30860 sys_catalog.cc:263] Verifying existing consensus state
I20260806 11:36:56.873637 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.193:44447
--local_ip_for_outbound_sockets=127.21.14.193
--tserver_master_addrs=127.21.14.254:45895
--webserver_port=40067
--webserver_interface=127.21.14.193
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab
--principal=kudu/127.21.14.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:42925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.21.14.193
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/krb5kdc/oryx_127.21.14.193.keytab with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 11:36:56.875526 30860 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fc5cd921ca240ab8220d493662a7f88: Bootstrap starting.
I20260806 11:36:56.895021 30860 log.cc:824] T 00000000000000000000000000000000 P 5fc5cd921ca240ab8220d493662a7f88: Log is configured to *not* fsync() on all Append() calls
I20260806 11:36:56.902695 30860 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fc5cd921ca240ab8220d493662a7f88: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 11:36:56.903206 30860 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fc5cd921ca240ab8220d493662a7f88: Bootstrap complete.
I20260806 11:36:56.908190 30860 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5fc5cd921ca240ab8220d493662a7f88 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fc5cd921ca240ab8220d493662a7f88" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 45895 } }
I20260806 11:36:56.908717 30860 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5fc5cd921ca240ab8220d493662a7f88 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fc5cd921ca240ab8220d493662a7f88, State: Initialized, Role: FOLLOWER
I20260806 11:36:56.909317 30860 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5fc5cd921ca240ab8220d493662a7f88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fc5cd921ca240ab8220d493662a7f88" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 45895 } }
I20260806 11:36:56.909504 30860 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5fc5cd921ca240ab8220d493662a7f88 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 11:36:56.909597 30860 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5fc5cd921ca240ab8220d493662a7f88 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 11:36:56.909735 30860 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5fc5cd921ca240ab8220d493662a7f88 [term 1 FOLLOWER]: Advancing to term 2
I20260806 11:36:56.912207 30860 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5fc5cd921ca240ab8220d493662a7f88 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fc5cd921ca240ab8220d493662a7f88" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 45895 } }
I20260806 11:36:56.912631 30860 leader_election.cc:302] T 00000000000000000000000000000000 P 5fc5cd921ca240ab8220d493662a7f88 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5fc5cd921ca240ab8220d493662a7f88; no voters:
I20260806 11:36:56.913028 30860 leader_election.cc:288] T 00000000000000000000000000000000 P 5fc5cd921ca240ab8220d493662a7f88 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 11:36:56.913144 30864 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5fc5cd921ca240ab8220d493662a7f88 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 11:36:56.913376 30864 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5fc5cd921ca240ab8220d493662a7f88 [term 2 LEADER]: Becoming Leader. State: Replica: 5fc5cd921ca240ab8220d493662a7f88, State: Running, Role: LEADER
I20260806 11:36:56.913718 30864 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5fc5cd921ca240ab8220d493662a7f88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fc5cd921ca240ab8220d493662a7f88" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 45895 } }
I20260806 11:36:56.914476 30860 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5fc5cd921ca240ab8220d493662a7f88 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 11:36:56.915252 30865 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fc5cd921ca240ab8220d493662a7f88 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "5fc5cd921ca240ab8220d493662a7f88" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fc5cd921ca240ab8220d493662a7f88" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 45895 } } }
I20260806 11:36:56.915402 30865 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fc5cd921ca240ab8220d493662a7f88 [sys.catalog]: This master's current role is: LEADER
I20260806 11:36:56.915761 30866 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fc5cd921ca240ab8220d493662a7f88 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5fc5cd921ca240ab8220d493662a7f88. Latest consensus state: current_term: 2 leader_uuid: "5fc5cd921ca240ab8220d493662a7f88" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fc5cd921ca240ab8220d493662a7f88" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 45895 } } }
I20260806 11:36:56.915853 30866 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fc5cd921ca240ab8220d493662a7f88 [sys.catalog]: This master's current role is: LEADER
I20260806 11:36:56.916502 30870 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 11:36:56.920761 30870 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 11:36:56.921715 30870 catalog_manager.cc:1259] Loaded cluster ID: 1f36bd5082aa4373ba20756619e62cfd
I20260806 11:36:56.921799 30870 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 11:36:56.924654 30870 catalog_manager.cc:1504] Loading token signing keys...
I20260806 11:36:56.925663 30870 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5fc5cd921ca240ab8220d493662a7f88: Loaded TSK: 0
I20260806 11:36:56.927346 30870 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 11:36:57.004292 30862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:36:57.004680 30862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:36:57.004768 30862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:36:57.009259 30862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:36:57.009387 30862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:36:57.009485 30862 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.193
I20260806 11:36:57.015693 30862 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:42925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/krb5kdc/oryx_127.21.14.193.keytab
--principal=oryx/127.21.14.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.193:44447
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.14.193
--webserver_port=40067
--tserver_master_addrs=127.21.14.254:45895
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30862
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.193
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:36:57.017102 30862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:36:57.018307 30862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:36:57.029928 30887 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:57.030035 30862 server_base.cc:1034] running on GCE node
W20260806 11:36:57.029922 30886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:57.030325 30889 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:57.030804 30862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:36:57.031741 30862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:36:57.032963 30862 hybrid_clock.cc:648] HybridClock initialized: now 1786016217032929 us; error 60 us; skew 500 ppm
Aug 06 11:36:57 dist-test-slave-7c35 krb5kdc[30228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016217, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.14.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:36:57.036592 30862 init.cc:377] Logged in from keytab as oryx/127.21.14.193@KRBTEST.COM (short username oryx)
I20260806 11:36:57.037649 30862 webserver.cc:460] Webserver started at http://127.21.14.193:40067/ using document root <none> and password file <none>
I20260806 11:36:57.038314 30862 fs_manager.cc:359] Metadata directory not provided
I20260806 11:36:57.038439 30862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:36:57.043452 30862 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 11:36:57.046262 30896 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:57.048097 30862 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 11:36:57.048281 30862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-0/wal
uuid: "6a622d7a17da41079e47dd97342f467e"
format_stamp: "Formatted at 2026-08-06 11:36:55 on dist-test-slave-7c35"
server_key: "1f66265cc9689d1018fc58dfc2128411"
server_key_iv: "5c53fdddb3d1e35cb0ea517676e1d3ce"
server_key_version: "encryptionkey@0"
I20260806 11:36:57.048982 30862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:36:57.077935 30862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:36:57.082053 30862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:36:57.082423 30862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:36:57.083290 30862 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:36:57.084352 30862 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:36:57.084436 30862 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:57.084517 30862 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:36:57.084566 30862 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:57.095094 30862 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.193:44447
I20260806 11:36:57.095130 31021 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.193:44447 every 8 connection(s)
I20260806 11:36:57.096493 30862 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
I20260806 11:36:57.103118 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 30862
I20260806 11:36:57.103643 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.194:35237
--local_ip_for_outbound_sockets=127.21.14.194
--tserver_master_addrs=127.21.14.254:45895
--webserver_port=46533
--webserver_interface=127.21.14.194
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab
--principal=kudu/127.21.14.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:42925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.21.14.194
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/krb5kdc/oryx_127.21.14.194.keytab with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 11:36:57 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.193@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 11:36:57 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.193@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
W20260806 11:36:57.114555 31024 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:36:57.099365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:45239 (local address 127.21.14.254:45895)
0806 11:36:57.100063 (+ 698us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:57.100086 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:57.100301 (+ 215us) server_negotiation.cc:407] Connection header received
0806 11:36:57.102039 (+ 1738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:57.102055 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:57.102464 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:57.103043 (+ 579us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:57.104947 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:57.106676 (+ 1729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:57.108254 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:57.108940 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:57.109162 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:57.113630 (+ 4468us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.14.193:45239: BlockingRecv error: recv got EOF from 127.21.14.193:45239 (error 108)
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":163,"threads_started":1}
I20260806 11:36:57.114590 31025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:57.099370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:45895 (local address 127.21.14.193:45239)
0806 11:36:57.100057 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:57.100122 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:57.101495 (+ 1373us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:57.103362 (+ 1867us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:57.103371 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:57.104080 (+ 709us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:57.104676 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:57.104696 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:57.106939 (+ 2243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:57.106943 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:57.108036 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:57.108062 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:57.109173 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:57.109178 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:57.109396 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:57.110333 (+ 937us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:57.110355 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:57.113326 (+ 2971us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.14.254:45895: Server oryx/127.21.14.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":196,"threads_started":1}
W20260806 11:36:57.115520 31022 heartbeater.cc:644] Failed to heartbeat to 127.21.14.254:45895 (0 consecutive failures): Not authorized: Failed to ping master at 127.21.14.254:45895: Client connection negotiation failed: client connection to 127.21.14.254:45895: Server oryx/127.21.14.254@KRBTEST.COM not found in Kerberos database
Aug 06 11:36:57 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.193@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 11:36:57 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.193@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 11:36:57.123706 31025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:57.116151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:45895 (local address 127.21.14.193:49527)
0806 11:36:57.116363 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:57.116383 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:57.116545 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:57.116942 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:57.116947 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:57.117180 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:57.117355 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:57.117365 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:57.118373 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:57.118377 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:57.119384 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:57.119400 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:57.120036 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:57.120040 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:57.120152 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:57.120823 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:57.120843 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:57.123510 (+ 2667us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.14.254:45895: Server oryx/127.21.14.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":92}
W20260806 11:36:57.123837 31024 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:36:57.116224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:49527 (local address 127.21.14.254:45895)
0806 11:36:57.116373 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:57.116377 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:57.116511 (+ 134us) server_negotiation.cc:407] Connection header received
0806 11:36:57.116728 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:57.116731 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:57.116791 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:57.116870 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:57.117663 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:57.118244 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:57.119612 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:57.119894 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:57.119996 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:57.123703 (+ 3707us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.14.193:49527: BlockingRecv error: recv got EOF from 127.21.14.193:49527 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 06 11:36:57 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.193@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 11:36:57 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.193@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 11:36:57.131237 31025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:57.124460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:45895 (local address 127.21.14.193:35977)
0806 11:36:57.124677 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:57.124691 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:57.124779 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:57.125140 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:57.125143 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:57.125346 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:57.125458 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:57.125464 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:57.126274 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:57.126281 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:57.127255 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:57.127271 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:57.127834 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:57.127837 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:57.127935 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:57.128630 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:57.128648 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:57.131081 (+ 2433us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.14.254:45895: Server oryx/127.21.14.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141}
W20260806 11:36:57.131520 31024 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:36:57.124622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:35977 (local address 127.21.14.254:45895)
0806 11:36:57.124758 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:57.124761 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:57.124777 (+ 16us) server_negotiation.cc:407] Connection header received
0806 11:36:57.124954 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:57.124958 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:57.125012 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:57.125095 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:57.125605 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:57.126151 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:57.127427 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:57.127682 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:57.127792 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:57.131326 (+ 3534us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.14.193:35977: BlockingRecv error: recv got EOF from 127.21.14.193:35977 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260806 11:36:57.131631 31022 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 11:36:57.255266 31026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:36:57.255605 31026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:36:57.255653 31026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:36:57.260340 31026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:36:57.260461 31026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:36:57.260533 31026 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.194
I20260806 11:36:57.267050 31026 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:42925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/krb5kdc/oryx_127.21.14.194.keytab
--principal=oryx/127.21.14.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.194:35237
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.14.194
--webserver_port=46533
--tserver_master_addrs=127.21.14.254:45895
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31026
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.194
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:36:57.268436 31026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:36:57.269924 31026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:36:57.281773 31033 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:57.281756 31035 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:57.281756 31032 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:57.282446 31026 server_base.cc:1034] running on GCE node
I20260806 11:36:57.283190 31026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:36:57.284029 31026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:36:57.285300 31026 hybrid_clock.cc:648] HybridClock initialized: now 1786016217285254 us; error 65 us; skew 500 ppm
Aug 06 11:36:57 dist-test-slave-7c35 krb5kdc[30228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016217, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.14.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:36:57.289307 31026 init.cc:377] Logged in from keytab as oryx/127.21.14.194@KRBTEST.COM (short username oryx)
I20260806 11:36:57.290431 31026 webserver.cc:460] Webserver started at http://127.21.14.194:46533/ using document root <none> and password file <none>
I20260806 11:36:57.291172 31026 fs_manager.cc:359] Metadata directory not provided
I20260806 11:36:57.291270 31026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:36:57.295996 31026 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 11:36:57.298545 31043 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:57.300264 31026 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 11:36:57.300459 31026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-1/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-1/wal
uuid: "cded16d898934ba09215b52922e630b2"
format_stamp: "Formatted at 2026-08-06 11:36:56 on dist-test-slave-7c35"
server_key: "0e3ef3810d9e44ebfb709b01f4c385ab"
server_key_iv: "3b1f8ea6cf91cc5fdbe27c1c61eeead7"
server_key_version: "encryptionkey@0"
I20260806 11:36:57.301141 31026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:36:57.328250 31026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:36:57.331972 31026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:36:57.332171 31026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:36:57.333036 31026 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:36:57.334138 31026 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:36:57.334193 31026 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:57.334263 31026 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:36:57.334306 31026 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:57.344796 31026 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.194:35237
I20260806 11:36:57.344817 31168 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.194:35237 every 8 connection(s)
I20260806 11:36:57.346134 31026 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
I20260806 11:36:57.354481 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 31026
I20260806 11:36:57.355099 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.195:43735
--local_ip_for_outbound_sockets=127.21.14.195
--tserver_master_addrs=127.21.14.254:45895
--webserver_port=33377
--webserver_interface=127.21.14.195
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab
--principal=kudu/127.21.14.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:42925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.21.14.195
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/krb5kdc/oryx_127.21.14.195.keytab with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 11:36:57 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.194@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 11:36:57 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.194@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
W20260806 11:36:57.361333 31024 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:36:57.348668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:47065 (local address 127.21.14.254:45895)
0806 11:36:57.348816 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:57.348820 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:57.350103 (+ 1283us) server_negotiation.cc:407] Connection header received
0806 11:36:57.351625 (+ 1522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:57.351629 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:57.351695 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:57.351796 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:57.353321 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:57.353937 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:57.356061 (+ 2124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:57.356305 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:57.356416 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:57.361195 (+ 4779us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.14.194:47065: BlockingRecv error: recv got EOF from 127.21.14.194:47065 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 11:36:57.362403 31171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:57.349135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:45895 (local address 127.21.14.194:47065)
0806 11:36:57.349860 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:57.349914 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:57.351275 (+ 1361us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:57.352013 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:57.352022 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:57.352486 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:57.353046 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:57.353062 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:57.354154 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:57.354158 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:57.355840 (+ 1682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:57.355861 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:57.356598 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:57.356606 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:57.356892 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:57.357776 (+ 884us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:57.357795 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:57.361019 (+ 3224us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.14.254:45895: Server oryx/127.21.14.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":158,"threads_started":1}
W20260806 11:36:57.363066 31169 heartbeater.cc:644] Failed to heartbeat to 127.21.14.254:45895 (0 consecutive failures): Not authorized: Failed to ping master at 127.21.14.254:45895: Client connection negotiation failed: client connection to 127.21.14.254:45895: Server oryx/127.21.14.254@KRBTEST.COM not found in Kerberos database
Aug 06 11:36:57 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.194@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 11:36:57 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.194@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 11:36:57.373842 31171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:57.363630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:45895 (local address 127.21.14.194:54917)
0806 11:36:57.363745 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:57.363760 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:57.363885 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:57.364203 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:57.364206 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:57.364401 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:57.364509 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:57.364515 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:57.365441 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:57.365447 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:57.366322 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:57.366336 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:57.367485 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:57.367491 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:57.367639 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:57.371050 (+ 3411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:57.371075 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:57.373599 (+ 2524us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.14.254:45895: Server oryx/127.21.14.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":33}
W20260806 11:36:57.374917 31024 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:36:57.363770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:54917 (local address 127.21.14.254:45895)
0806 11:36:57.363931 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:57.363938 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:57.363956 (+ 18us) server_negotiation.cc:407] Connection header received
0806 11:36:57.364019 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:57.364021 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:57.364062 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:57.364181 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:57.364647 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:57.365286 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:57.366597 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:57.367135 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:57.367250 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:57.374739 (+ 7489us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.14.194:54917: BlockingRecv error: recv got EOF from 127.21.14.194:54917 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 06 11:36:57 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.194@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 11:36:57 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.194@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 11:36:57.381335 31171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:57.374651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:45895 (local address 127.21.14.194:40665)
0806 11:36:57.374816 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:57.374833 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:57.374979 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:57.375577 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:57.375581 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:57.375809 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:57.375961 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:57.375971 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:57.377141 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:57.377144 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:57.377703 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:57.377712 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:57.378354 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:57.378357 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:57.378425 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:57.379038 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:57.379051 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:57.381183 (+ 2132us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.14.254:45895: Server oryx/127.21.14.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":72}
W20260806 11:36:57.381587 31169 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 11:36:57.381800 31024 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:36:57.375207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:40665 (local address 127.21.14.254:45895)
0806 11:36:57.375313 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:57.375319 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:57.375337 (+ 18us) server_negotiation.cc:407] Connection header received
0806 11:36:57.375390 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:57.375392 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:57.375450 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:57.375551 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:57.376131 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:57.377024 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:57.377842 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:57.378214 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:57.378329 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:57.381361 (+ 3032us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.14.194:40665: BlockingRecv error: recv got EOF from 127.21.14.194:40665 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
W20260806 11:36:57.493397 31172 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:36:57.493808 31172 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:36:57.493893 31172 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:36:57.498260 31172 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:36:57.498401 31172 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:36:57.498543 31172 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.195
I20260806 11:36:57.504860 31172 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:42925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/krb5kdc/oryx_127.21.14.195.keytab
--principal=oryx/127.21.14.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.195:43735
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.14.195
--webserver_port=33377
--tserver_master_addrs=127.21.14.254:45895
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31172
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.195
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:36:57.506194 31172 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:36:57.507428 31172 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:36:57.519054 31182 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:57.519075 31179 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:36:57.519111 31180 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:36:57.519533 31172 server_base.cc:1034] running on GCE node
I20260806 11:36:57.520073 31172 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:36:57.520886 31172 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:36:57.522120 31172 hybrid_clock.cc:648] HybridClock initialized: now 1786016217522100 us; error 38 us; skew 500 ppm
Aug 06 11:36:57 dist-test-slave-7c35 krb5kdc[30228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016217, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.14.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:36:57.525931 31172 init.cc:377] Logged in from keytab as oryx/127.21.14.195@KRBTEST.COM (short username oryx)
I20260806 11:36:57.527645 31172 webserver.cc:460] Webserver started at http://127.21.14.195:33377/ using document root <none> and password file <none>
I20260806 11:36:57.528357 31172 fs_manager.cc:359] Metadata directory not provided
I20260806 11:36:57.528493 31172 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:36:57.534687 31172 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 11:36:57.537451 31189 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:57.539263 31172 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 11:36:57.539454 31172 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-2/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-2/wal
uuid: "781b6728bbc34238b584293ee898c245"
format_stamp: "Formatted at 2026-08-06 11:36:56 on dist-test-slave-7c35"
server_key: "3b38719225212651d9234b55061db409"
server_key_iv: "90faa2167f89c8651667acbea6d4c318"
server_key_version: "encryptionkey@0"
I20260806 11:36:57.540145 31172 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:36:57.560323 31172 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:36:57.564530 31172 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:36:57.564772 31172 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:36:57.565470 31172 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:36:57.566443 31172 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:36:57.566517 31172 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:57.566619 31172 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:36:57.566659 31172 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:36:57.577244 31172 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.195:43735
I20260806 11:36:57.577294 31312 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.195:43735 every 8 connection(s)
I20260806 11:36:57.578428 31172 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
I20260806 11:36:57.585644 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 31172
Aug 06 11:36:57 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.195@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 11:36:57 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.195@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
W20260806 11:36:57.592973 31024 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:36:57.580637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:50373 (local address 127.21.14.254:45895)
0806 11:36:57.580809 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:57.580814 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:57.581905 (+ 1091us) server_negotiation.cc:407] Connection header received
0806 11:36:57.583468 (+ 1563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:57.583472 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:57.583529 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:57.583620 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:57.585075 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:57.585616 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:57.587254 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:57.587640 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:57.587743 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:57.592760 (+ 5017us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.14.195:50373: BlockingRecv error: recv got EOF from 127.21.14.195:50373 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 11:36:57.593308 31315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:57.581159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:45895 (local address 127.21.14.195:50373)
0806 11:36:57.581719 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:57.581768 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:57.583163 (+ 1395us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:57.583902 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:57.583910 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:57.584324 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:57.584889 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:57.584905 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:57.585789 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:57.585794 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:57.587088 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:57.587102 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:57.587839 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:57.587845 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:57.588122 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:57.588950 (+ 828us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:57.588969 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:57.592298 (+ 3329us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.14.254:45895: Server oryx/127.21.14.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":133,"threads_started":1}
W20260806 11:36:57.594063 31313 heartbeater.cc:644] Failed to heartbeat to 127.21.14.254:45895 (0 consecutive failures): Not authorized: Failed to ping master at 127.21.14.254:45895: Client connection negotiation failed: client connection to 127.21.14.254:45895: Server oryx/127.21.14.254@KRBTEST.COM not found in Kerberos database
I20260806 11:36:57.599128 31318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:57.586985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50694 (local address 127.21.14.254:45895)
0806 11:36:57.587346 (+ 361us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:57.587349 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:57.587370 (+ 21us) server_negotiation.cc:407] Connection header received
0806 11:36:57.587404 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:57.587406 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:57.587446 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:57.587560 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:57.588565 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:57.589259 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:57.590637 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:57.591116 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:57.591250 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:57.593842 (+ 2592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:36:57.593850 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:36:57.593904 (+ 54us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:36:57.593947 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:36:57.596386 (+ 2439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:57.597003 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:57.597014 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:57.597041 (+ 27us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:57.597200 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:36:57.597553 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:36:57.597555 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:36:57.597556 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:36:57.598019 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:36:57.598229 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0806 11:36:57.598571 (+ 342us) server_negotiation.cc:299] Negotiation successful
0806 11:36:57.598957 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"spinlock_wait_cycles":17280,"thread_start_us":211,"threads_started":1}
Aug 06 11:36:57 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.195@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 11:36:57 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.195@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 11:36:57.602376 31315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:57.594684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:45895 (local address 127.21.14.195:43973)
0806 11:36:57.594878 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:57.594895 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:57.594989 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:57.595362 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:57.595366 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:57.595586 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:57.595728 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:57.595737 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:57.597025 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:57.597029 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:57.597851 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:57.597866 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:57.598497 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:57.598503 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:57.598617 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:57.599225 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:57.599265 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:57.602218 (+ 2953us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.14.254:45895: Server oryx/127.21.14.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":72}
W20260806 11:36:57.602705 31024 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:36:57.594795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:43973 (local address 127.21.14.254:45895)
0806 11:36:57.594941 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:57.594950 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:57.594963 (+ 13us) server_negotiation.cc:407] Connection header received
0806 11:36:57.595121 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:57.595126 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:57.595190 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:57.595283 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:57.596040 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:57.596701 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:57.597988 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:57.598358 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:57.598464 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:57.602407 (+ 3943us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.14.195:43973: BlockingRecv error: recv got EOF from 127.21.14.195:43973 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 06 11:36:57 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.195@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 11:36:57 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.195@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 11:36:57.609254 31315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:57.603457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:45895 (local address 127.21.14.195:38657)
0806 11:36:57.603650 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:57.603668 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:57.603779 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:57.604102 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:57.604104 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:57.604266 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:57.604356 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:57.604362 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:57.605230 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:57.605233 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:57.605785 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:57.605793 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:57.606337 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:57.606340 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:57.606406 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:57.607061 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:57.607074 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:57.609129 (+ 2055us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.14.254:45895: Server oryx/127.21.14.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":100}
W20260806 11:36:57.609647 31313 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 11:36:57.609745 31024 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:36:57.603617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:38657 (local address 127.21.14.254:45895)
0806 11:36:57.603835 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:57.603838 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:57.603851 (+ 13us) server_negotiation.cc:407] Connection header received
0806 11:36:57.603886 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:57.603888 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:57.603928 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:57.603996 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:57.604514 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:57.605074 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:57.605945 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:57.606174 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:57.606260 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:57.609597 (+ 3337us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.14.195:38657: BlockingRecv error: recv got EOF from 127.21.14.195:38657 (error 108)
Metrics: {"server-negotiator.queue_time_us":145}
Aug 06 11:36:58 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.193@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 11:36:58 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.193@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 11:36:58.138573 31320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:58.132393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:45895 (local address 127.21.14.193:37051)
0806 11:36:58.132708 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:58.132727 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:58.132852 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:58.133184 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:58.133187 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:58.133406 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:58.133508 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:58.133514 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:58.134323 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:58.134328 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:58.134949 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:58.134960 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:58.135494 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:58.135498 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:58.135589 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:58.136221 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:58.136238 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:58.138379 (+ 2141us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.14.254:45895: Server oryx/127.21.14.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":96,"threads_started":1}
W20260806 11:36:58.138911 31321 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:36:58.132498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:37051 (local address 127.21.14.254:45895)
0806 11:36:58.132833 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:58.132838 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:58.132854 (+ 16us) server_negotiation.cc:407] Connection header received
0806 11:36:58.132999 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:58.133002 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:58.133061 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:58.133159 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:58.133641 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:58.134199 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:58.135089 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:58.135339 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:58.135480 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:58.138572 (+ 3092us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.14.193:37051: BlockingRecv error: recv got EOF from 127.21.14.193:37051 (error 108)
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":87,"threads_started":1}
Aug 06 11:36:58 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.194@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 11:36:58 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.194@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 11:36:58.389031 31322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:58.382323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:45895 (local address 127.21.14.194:32833)
0806 11:36:58.382780 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:58.382795 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:58.383033 (+ 238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:58.383366 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:58.383370 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:58.383620 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:58.383728 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:58.383735 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:58.384613 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:58.384618 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:58.385299 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:58.385309 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:58.385857 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:58.385861 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:58.385972 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:58.386650 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:58.386664 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:58.388800 (+ 2136us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.14.254:45895: Server oryx/127.21.14.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":104,"threads_started":1}
W20260806 11:36:58.389118 31321 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:36:58.382503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:32833 (local address 127.21.14.254:45895)
0806 11:36:58.382715 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:58.382721 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:58.382912 (+ 191us) server_negotiation.cc:407] Connection header received
0806 11:36:58.383164 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:58.383168 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:58.383232 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:58.383335 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:58.383873 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:58.384486 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:58.385463 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:58.385721 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:58.385845 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:58.388979 (+ 3134us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.14.194:32833: BlockingRecv error: recv got EOF from 127.21.14.194:32833 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
Aug 06 11:36:58 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.195@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 11:36:58 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.195@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 11:36:58.618175 31323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:58.610596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:45895 (local address 127.21.14.195:58247)
0806 11:36:58.611069 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:58.611089 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:58.611247 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:58.611653 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:58.611655 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:58.611909 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:58.612086 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:58.612097 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:58.613038 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:58.613042 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:58.613682 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:58.613724 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:58.614580 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:58.614583 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:58.614705 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:58.615588 (+ 883us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:58.615608 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:58.618020 (+ 2412us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.14.254:45895: Server oryx/127.21.14.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":95,"threads_started":1}
W20260806 11:36:58.618433 31321 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:36:58.610677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:58247 (local address 127.21.14.254:45895)
0806 11:36:58.610871 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:58.610876 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:58.611220 (+ 344us) server_negotiation.cc:407] Connection header received
0806 11:36:58.611385 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:58.611389 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:58.611461 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:58.611585 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:58.612309 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:58.612884 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:58.613929 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:58.614325 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:58.614403 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:58.618267 (+ 3864us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.14.195:58247: BlockingRecv error: recv got EOF from 127.21.14.195:58247 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 06 11:36:59 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.193@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 11:36:59 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.193@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 11:36:59.146968 31324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:59.139770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:45895 (local address 127.21.14.193:40801)
0806 11:36:59.140292 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:59.140310 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:59.140455 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:59.140751 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:59.140754 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:59.141007 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:59.141167 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:59.141176 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:59.142015 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:59.142019 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:59.142613 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:59.142622 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:59.143224 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:59.143228 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:59.143333 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:59.144126 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:59.144147 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:59.146713 (+ 2566us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.14.254:45895: Server oryx/127.21.14.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":141,"threads_started":1}
W20260806 11:36:59.147070 31326 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:36:59.140010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:40801 (local address 127.21.14.254:45895)
0806 11:36:59.140336 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:59.140340 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:59.140387 (+ 47us) server_negotiation.cc:407] Connection header received
0806 11:36:59.140551 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:59.140556 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:59.140615 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:59.140724 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:59.141328 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:59.141888 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:59.142758 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:59.143083 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:59.143202 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:59.146921 (+ 3719us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.14.193:40801: BlockingRecv error: recv got EOF from 127.21.14.193:40801 (error 108)
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":109,"threads_started":1}
Aug 06 11:36:59 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.194@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 11:36:59 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.194@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 11:36:59.396092 31327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:59.390158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:45895 (local address 127.21.14.194:59371)
0806 11:36:59.390524 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:59.390537 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:59.390656 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:59.390952 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:59.390954 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:59.391164 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:59.391270 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:59.391276 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:59.392105 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:59.392108 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:59.392672 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:59.392680 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:59.393142 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:59.393145 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:59.393210 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:59.393761 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:59.393773 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:59.395927 (+ 2154us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.14.254:45895: Server oryx/127.21.14.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":126,"threads_started":1}
W20260806 11:36:59.396283 31326 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:36:59.390280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:59371 (local address 127.21.14.254:45895)
0806 11:36:59.390483 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:59.390487 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:59.390620 (+ 133us) server_negotiation.cc:407] Connection header received
0806 11:36:59.390741 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:59.390743 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:59.390793 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:59.390930 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:59.391396 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:59.391974 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:59.392788 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:59.393023 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:59.393107 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:59.396110 (+ 3003us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.14.194:59371: BlockingRecv error: recv got EOF from 127.21.14.194:59371 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 06 11:36:59 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.195@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 11:36:59 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.195@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 11:36:59.627566 31328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:36:59.619351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:45895 (local address 127.21.14.195:57791)
0806 11:36:59.619731 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0806 11:36:59.619745 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 11:36:59.619871 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:36:59.620240 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:36:59.620243 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:36:59.620473 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:36:59.620571 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:59.620577 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:59.621780 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:59.621784 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:59.622746 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:36:59.622761 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:36:59.624540 (+ 1779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:36:59.624546 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:36:59.624652 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:59.625214 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:36:59.625227 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:36:59.627391 (+ 2164us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.14.254:45895: Server oryx/127.21.14.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":117,"threads_started":1}
W20260806 11:36:59.627764 31326 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:36:59.619508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:57791 (local address 127.21.14.254:45895)
0806 11:36:59.619699 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 11:36:59.619704 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:36:59.619829 (+ 125us) server_negotiation.cc:407] Connection header received
0806 11:36:59.620041 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:36:59.620046 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:36:59.620107 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:36:59.620213 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:36:59.620758 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:59.621615 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:59.622943 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:36:59.624341 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:36:59.624500 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:36:59.627605 (+ 3105us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.14.195:57791: BlockingRecv error: recv got EOF from 127.21.14.195:57791 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
Aug 06 11:37:00 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.193@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 11:37:00 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.193@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 11:37:00.156291 31329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:37:00.148162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:45895 (local address 127.21.14.193:40219)
0806 11:37:00.148518 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0806 11:37:00.148534 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 11:37:00.148637 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:37:00.149086 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:37:00.149090 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:37:00.149466 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:37:00.149754 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:00.149776 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:00.151003 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:00.151008 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:00.151772 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:00.151787 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:00.152669 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:00.152673 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:00.152853 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:00.153699 (+ 846us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:37:00.153724 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:37:00.156149 (+ 2425us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.14.254:45895: Server oryx/127.21.14.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":149,"threads_started":1}
W20260806 11:37:00.156626 31330 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:37:00.148390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:40219 (local address 127.21.14.254:45895)
0806 11:37:00.148715 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0806 11:37:00.148719 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:37:00.148738 (+ 19us) server_negotiation.cc:407] Connection header received
0806 11:37:00.148787 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:37:00.148790 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:37:00.148832 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:37:00.148975 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:37:00.149942 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:00.150707 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:00.151964 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:00.152458 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:00.152586 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:00.156450 (+ 3864us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.14.193:40219: BlockingRecv error: recv got EOF from 127.21.14.193:40219 (error 108)
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":79,"threads_started":1}
Aug 06 11:37:00 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.194@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 11:37:00 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.194@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 11:37:00.403918 31331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:37:00.397193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:45895 (local address 127.21.14.194:40933)
0806 11:37:00.397524 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0806 11:37:00.397538 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 11:37:00.397656 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:37:00.398000 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:37:00.398002 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:37:00.398209 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:37:00.398314 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:00.398320 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:00.399356 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:00.399367 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:00.399972 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:00.399982 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:00.400531 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:00.400545 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:00.400622 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:00.401285 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:37:00.401304 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:37:00.403787 (+ 2483us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.14.254:45895: Server oryx/127.21.14.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":123,"threads_started":1}
W20260806 11:37:00.404129 31330 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:37:00.397331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:40933 (local address 127.21.14.254:45895)
0806 11:37:00.397558 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0806 11:37:00.397562 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:37:00.397604 (+ 42us) server_negotiation.cc:407] Connection header received
0806 11:37:00.397768 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:37:00.397771 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:37:00.397824 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:37:00.397935 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:37:00.398479 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:00.399203 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:00.400138 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:00.400379 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:00.400484 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:00.403941 (+ 3457us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.14.194:40933: BlockingRecv error: recv got EOF from 127.21.14.194:40933 (error 108)
Metrics: {"server-negotiator.queue_time_us":121}
Aug 06 11:37:00 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.195@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 11:37:00 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.195@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 11:37:00.636202 31332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:37:00.628529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:45895 (local address 127.21.14.195:53717)
0806 11:37:00.628893 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0806 11:37:00.628915 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 11:37:00.629049 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:37:00.629522 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:37:00.629525 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:37:00.629758 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:37:00.629866 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:00.629873 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:00.630720 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:00.630724 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:00.631475 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:00.631488 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:00.632161 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:00.632166 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:00.632265 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:00.633417 (+ 1152us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:37:00.633571 (+ 154us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:37:00.636064 (+ 2493us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.14.254:45895: Server oryx/127.21.14.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":144,"threads_started":1}
W20260806 11:37:00.636439 31330 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:37:00.628712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:53717 (local address 127.21.14.254:45895)
0806 11:37:00.628866 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 11:37:00.628871 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:37:00.629001 (+ 130us) server_negotiation.cc:407] Connection header received
0806 11:37:00.629296 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:37:00.629302 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:37:00.629370 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:37:00.629494 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:37:00.630000 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:00.630582 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:00.631638 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:00.631982 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:00.632107 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:00.636301 (+ 4194us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.14.195:53717: BlockingRecv error: recv got EOF from 127.21.14.195:53717 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 06 11:37:01 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.193@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 11:37:01 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.193@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 11:37:01.171823 31333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:37:01.157378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:45895 (local address 127.21.14.193:57037)
0806 11:37:01.157760 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0806 11:37:01.157774 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 11:37:01.157874 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:37:01.158338 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:37:01.158341 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:37:01.158585 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:37:01.158703 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:01.158722 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:01.159621 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:01.159624 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:01.160266 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:01.160277 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:01.160976 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:01.160979 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:01.161063 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:01.168909 (+ 7846us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:37:01.168933 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:37:01.171683 (+ 2750us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.14.254:45895: Server oryx/127.21.14.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":137,"threads_started":1}
W20260806 11:37:01.172106 31334 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:37:01.157552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:57037 (local address 127.21.14.254:45895)
0806 11:37:01.157862 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0806 11:37:01.157865 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 11:37:01.157879 (+ 14us) server_negotiation.cc:407] Connection header received
0806 11:37:01.157948 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:37:01.157951 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:37:01.158004 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:37:01.158083 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:37:01.158894 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:01.159478 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:01.160444 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:01.160728 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:01.160846 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:01.171925 (+ 11079us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.14.193:57037: BlockingRecv error: recv got EOF from 127.21.14.193:57037 (error 108)
Metrics: {"server-negotiator.queue_time_us":232,"thread_start_us":152,"threads_started":1}
Aug 06 11:37:01 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.194@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 11:37:01 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.194@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 11:37:01.411574 31335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:37:01.404945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:45895 (local address 127.21.14.194:49989)
0806 11:37:01.405226 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0806 11:37:01.405249 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 11:37:01.405384 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:37:01.405709 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:37:01.405712 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:37:01.406046 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:37:01.406162 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:01.406168 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:01.407087 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:01.407091 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:01.407672 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:01.407682 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:01.408192 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:01.408196 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:01.408289 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:01.409000 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:37:01.409052 (+ 52us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:37:01.411365 (+ 2313us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.14.254:45895: Server oryx/127.21.14.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":102,"threads_started":1}
W20260806 11:37:01.411685 31334 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:37:01.405053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:49989 (local address 127.21.14.254:45895)
0806 11:37:01.405205 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 11:37:01.405211 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 11:37:01.405335 (+ 124us) server_negotiation.cc:407] Connection header received
0806 11:37:01.405485 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:37:01.405506 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:37:01.405570 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:37:01.405685 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:37:01.406292 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:01.406975 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:01.407810 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:01.408058 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:01.408172 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:01.411552 (+ 3380us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.14.194:49989: BlockingRecv error: recv got EOF from 127.21.14.194:49989 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 06 11:37:01 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.195@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 11:37:01 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.195@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 11:37:01.643991 31336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:37:01.637269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:45895 (local address 127.21.14.195:40419)
0806 11:37:01.637566 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0806 11:37:01.637586 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 11:37:01.637716 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:37:01.638093 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:37:01.638096 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:37:01.638292 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:37:01.638397 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:01.638403 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:01.639420 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:01.639424 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:01.640083 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:01.640098 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:01.640704 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:01.640709 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:01.640808 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:01.641454 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:37:01.641478 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:37:01.643855 (+ 2377us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.14.254:45895: Server oryx/127.21.14.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":122,"threads_started":1}
W20260806 11:37:01.644423 31334 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:37:01.637381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:40419 (local address 127.21.14.254:45895)
0806 11:37:01.637561 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0806 11:37:01.637565 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:37:01.637706 (+ 141us) server_negotiation.cc:407] Connection header received
0806 11:37:01.637836 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:37:01.637839 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:37:01.637894 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:37:01.637988 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:37:01.638562 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:01.639242 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:01.640243 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:01.640529 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:01.640664 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:01.644065 (+ 3401us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.14.195:40419: BlockingRecv error: recv got EOF from 127.21.14.195:40419 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 06 11:37:02 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.193@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 11:37:02 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.193@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 11:37:02.180014 31337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:37:02.172902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:45895 (local address 127.21.14.193:53505)
0806 11:37:02.173370 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0806 11:37:02.173386 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 11:37:02.173503 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:37:02.173886 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:37:02.173889 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:37:02.174099 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:37:02.174202 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:02.174209 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:02.175246 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:02.175250 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:02.176068 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:02.176081 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:02.176964 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:02.176968 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:02.177065 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:02.177685 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:37:02.177697 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:37:02.179880 (+ 2183us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.14.254:45895: Server oryx/127.21.14.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":156,"threads_started":1}
W20260806 11:37:02.180639 31338 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:37:02.173103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:53505 (local address 127.21.14.254:45895)
0806 11:37:02.173405 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0806 11:37:02.173409 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:37:02.173450 (+ 41us) server_negotiation.cc:407] Connection header received
0806 11:37:02.173634 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:37:02.173637 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:37:02.173685 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:37:02.173779 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:37:02.174356 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:02.175091 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:02.176335 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:02.176716 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:02.176849 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:02.180475 (+ 3626us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.14.193:53505: BlockingRecv error: recv got EOF from 127.21.14.193:53505 (error 108)
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":93,"threads_started":1}
Aug 06 11:37:02 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.194@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 11:37:02 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.194@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 11:37:02.418938 31339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:37:02.412547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:45895 (local address 127.21.14.194:58377)
0806 11:37:02.412886 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0806 11:37:02.412903 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 11:37:02.413052 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:37:02.413325 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:37:02.413328 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:37:02.413561 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:37:02.413658 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:02.413664 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:02.414450 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:02.414454 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:02.415211 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:02.415222 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:02.415761 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:02.415765 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:02.415855 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:02.416521 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:37:02.416535 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:37:02.418711 (+ 2176us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.14.254:45895: Server oryx/127.21.14.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":116,"threads_started":1}
W20260806 11:37:02.419034 31338 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:37:02.412747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:58377 (local address 127.21.14.254:45895)
0806 11:37:02.412887 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 11:37:02.412893 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 11:37:02.412981 (+ 88us) server_negotiation.cc:407] Connection header received
0806 11:37:02.413160 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:37:02.413166 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:37:02.413217 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:37:02.413304 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:37:02.413783 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:02.414329 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:02.415342 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:02.415581 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:02.415746 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:02.418902 (+ 3156us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.14.194:58377: BlockingRecv error: recv got EOF from 127.21.14.194:58377 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 06 11:37:02 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.195@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 11:37:02 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.195@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 11:37:02.653812 31340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:37:02.645067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:45895 (local address 127.21.14.195:53767)
0806 11:37:02.645377 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0806 11:37:02.645392 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 11:37:02.645534 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:37:02.645881 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:37:02.645884 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:37:02.646339 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:37:02.646494 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:02.646507 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:02.647639 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:02.647646 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:02.648664 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:02.648678 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:02.649259 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:02.649341 (+ 82us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:02.649483 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:02.650395 (+ 912us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:37:02.650410 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:37:02.653625 (+ 3215us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.14.254:45895: Server oryx/127.21.14.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":76,"threads_started":1}
W20260806 11:37:02.654109 31338 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:37:02.645269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:53767 (local address 127.21.14.254:45895)
0806 11:37:02.645446 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0806 11:37:02.645449 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 11:37:02.645467 (+ 18us) server_negotiation.cc:407] Connection header received
0806 11:37:02.645688 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:37:02.645690 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:37:02.645744 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:37:02.645816 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:37:02.646694 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:02.647493 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:02.648857 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:02.649117 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:02.649210 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:02.653971 (+ 4761us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.14.195:53767: BlockingRecv error: recv got EOF from 127.21.14.195:53767 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 06 11:37:03 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.193@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 11:37:03 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.193@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 11:37:03.187776 31341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:37:03.180973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:45895 (local address 127.21.14.193:40085)
0806 11:37:03.181355 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0806 11:37:03.181373 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 11:37:03.181499 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:37:03.181852 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:37:03.181854 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:37:03.182060 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:37:03.182164 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:03.182170 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:03.183140 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:03.183143 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:03.183763 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:03.183772 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:03.184471 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:03.184474 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:03.184552 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:03.185191 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:37:03.185204 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:37:03.187565 (+ 2361us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.14.254:45895: Server oryx/127.21.14.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":82,"threads_started":1}
W20260806 11:37:03.187947 31342 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:37:03.181208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:40085 (local address 127.21.14.254:45895)
0806 11:37:03.181505 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0806 11:37:03.181508 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 11:37:03.181524 (+ 16us) server_negotiation.cc:407] Connection header received
0806 11:37:03.181589 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:37:03.181592 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:37:03.181636 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:37:03.181723 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:37:03.182328 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:03.182935 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:03.183946 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:03.184310 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:03.184416 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:03.187803 (+ 3387us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.14.193:40085: BlockingRecv error: recv got EOF from 127.21.14.193:40085 (error 108)
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":89,"threads_started":1}
Aug 06 11:37:03 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.194@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 11:37:03 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.194@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 11:37:03.426527 31343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:37:03.420012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:45895 (local address 127.21.14.194:45121)
0806 11:37:03.420395 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0806 11:37:03.420413 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 11:37:03.420566 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:37:03.420870 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:37:03.420874 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:37:03.421103 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:37:03.421225 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:03.421232 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:03.422068 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:03.422072 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:03.422652 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:03.422660 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:03.423227 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:03.423231 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:03.423349 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:03.423955 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:37:03.423971 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:37:03.426315 (+ 2344us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.14.254:45895: Server oryx/127.21.14.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":100,"threads_started":1}
W20260806 11:37:03.426623 31342 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:37:03.420183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:45121 (local address 127.21.14.254:45895)
0806 11:37:03.420361 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0806 11:37:03.420366 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:37:03.420513 (+ 147us) server_negotiation.cc:407] Connection header received
0806 11:37:03.420689 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:37:03.420692 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:37:03.420748 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:37:03.420854 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:37:03.421368 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:03.421937 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:03.422782 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:03.423089 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:03.423202 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:03.426498 (+ 3296us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.14.194:45121: BlockingRecv error: recv got EOF from 127.21.14.194:45121 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 06 11:37:03 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.195@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 11:37:03 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.195@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 11:37:03.660563 31344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:37:03.654730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:45895 (local address 127.21.14.195:33469)
0806 11:37:03.655083 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0806 11:37:03.655095 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 11:37:03.655192 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:37:03.655527 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:37:03.655532 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:37:03.655743 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:37:03.655840 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:03.655846 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:03.656768 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:03.656771 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:03.657350 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:03.657359 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:03.657828 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:03.657831 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:03.657894 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:03.658466 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:37:03.658478 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:37:03.660423 (+ 1945us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.14.254:45895: Server oryx/127.21.14.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":125,"threads_started":1}
W20260806 11:37:03.660820 31342 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:37:03.654882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:33469 (local address 127.21.14.254:45895)
0806 11:37:03.655033 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 11:37:03.655037 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:37:03.655170 (+ 133us) server_negotiation.cc:407] Connection header received
0806 11:37:03.655264 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:37:03.655267 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:37:03.655332 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:37:03.655423 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:37:03.655980 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:03.656652 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:03.657461 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:03.657720 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:03.657803 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:03.660689 (+ 2886us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.14.195:33469: BlockingRecv error: recv got EOF from 127.21.14.195:33469 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 06 11:37:04 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.193@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 11:37:04 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.193@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 11:37:04.195456 31345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:37:04.188885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:45895 (local address 127.21.14.193:35165)
0806 11:37:04.189271 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0806 11:37:04.189286 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 11:37:04.189384 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:37:04.189663 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:37:04.189665 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:37:04.189853 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:37:04.189979 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:04.189985 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:04.190835 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:04.190838 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:04.191586 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:04.191599 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:04.192110 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:04.192113 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:04.192273 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:04.192878 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:37:04.192891 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:37:04.195314 (+ 2423us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.14.254:45895: Server oryx/127.21.14.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":129,"threads_started":1}
W20260806 11:37:04.195760 31346 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:37:04.189131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:35165 (local address 127.21.14.254:45895)
0806 11:37:04.189431 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0806 11:37:04.189434 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 11:37:04.189447 (+ 13us) server_negotiation.cc:407] Connection header received
0806 11:37:04.189488 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:37:04.189490 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:37:04.189533 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:37:04.189619 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:37:04.190170 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:04.190723 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:04.191731 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:04.191968 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:04.192079 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:04.195590 (+ 3511us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.14.193:35165: BlockingRecv error: recv got EOF from 127.21.14.193:35165 (error 108)
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":139,"threads_started":1}
Aug 06 11:37:04 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.194@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 11:37:04 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.194@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 11:37:04.434221 31348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:37:04.427530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:45895 (local address 127.21.14.194:36675)
0806 11:37:04.427916 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0806 11:37:04.427929 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 11:37:04.428085 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:37:04.428566 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:37:04.428570 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:37:04.428822 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:37:04.428947 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:04.428954 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:04.429814 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:04.429818 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:04.430451 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:04.430461 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:04.431119 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:04.431123 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:04.431199 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:04.431799 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:37:04.431811 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:37:04.434082 (+ 2271us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.14.254:45895: Server oryx/127.21.14.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":130,"threads_started":1}
W20260806 11:37:04.434505 31346 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:37:04.427584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:36675 (local address 127.21.14.254:45895)
0806 11:37:04.427925 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0806 11:37:04.427931 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 11:37:04.427994 (+ 63us) server_negotiation.cc:407] Connection header received
0806 11:37:04.428252 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:37:04.428256 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:37:04.428312 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:37:04.428415 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:37:04.429138 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:04.429682 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:04.430638 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:04.430962 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:04.431067 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:04.434372 (+ 3305us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.14.194:36675: BlockingRecv error: recv got EOF from 127.21.14.194:36675 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 06 11:37:04 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.195@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 11:37:04 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.195@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 11:37:04.668037 31349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:37:04.661641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:45895 (local address 127.21.14.195:59355)
0806 11:37:04.661988 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0806 11:37:04.662003 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 11:37:04.662119 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:37:04.662372 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:37:04.662374 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:37:04.662576 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:37:04.662675 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:04.662681 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:04.663551 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:04.663554 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:04.664121 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:04.664129 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:04.664753 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:04.664757 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:04.664841 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:04.665553 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:37:04.665572 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:37:04.667902 (+ 2330us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.14.254:45895: Server oryx/127.21.14.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":99,"threads_started":1}
W20260806 11:37:04.668298 31346 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:37:04.661687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:59355 (local address 127.21.14.254:45895)
0806 11:37:04.661857 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 11:37:04.661862 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:37:04.662085 (+ 223us) server_negotiation.cc:407] Connection header received
0806 11:37:04.662211 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:37:04.662213 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:37:04.662266 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:37:04.662332 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:37:04.662789 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:04.663423 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:04.664294 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:04.664577 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:04.664703 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:04.668176 (+ 3473us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.14.195:59355: BlockingRecv error: recv got EOF from 127.21.14.195:59355 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 06 11:37:05 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.193@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 11:37:05 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.193@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 11:37:05.204020 31350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:37:05.196883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:45895 (local address 127.21.14.193:49785)
0806 11:37:05.197284 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0806 11:37:05.197299 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 11:37:05.197412 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:37:05.197683 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:37:05.197685 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:37:05.197872 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:37:05.197976 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:05.197981 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:05.199139 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:05.199146 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:05.199786 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:05.199799 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:05.200625 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:05.200632 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:05.200729 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:05.201497 (+ 768us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:37:05.201510 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:37:05.203841 (+ 2331us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.14.254:45895: Server oryx/127.21.14.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":141,"threads_started":1}
W20260806 11:37:05.204161 31351 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:37:05.197149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:49785 (local address 127.21.14.254:45895)
0806 11:37:05.197421 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0806 11:37:05.197425 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:37:05.197463 (+ 38us) server_negotiation.cc:407] Connection header received
0806 11:37:05.197511 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:37:05.197513 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:37:05.197558 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:37:05.197632 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:37:05.198132 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:05.198915 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:05.200013 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:05.200412 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:05.200540 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:05.204013 (+ 3473us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.14.193:49785: BlockingRecv error: recv got EOF from 127.21.14.193:49785 (error 108)
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":102,"threads_started":1}
Aug 06 11:37:05 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.194@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 11:37:05 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.194@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 11:37:05.441972 31352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:37:05.435233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:45895 (local address 127.21.14.194:33519)
0806 11:37:05.435572 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0806 11:37:05.435587 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 11:37:05.435706 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:37:05.436049 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:37:05.436052 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:37:05.436246 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:37:05.436348 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:05.436354 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:05.437327 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:05.437330 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:05.437968 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:05.437979 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:05.438577 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:05.438581 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:05.438689 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:05.439417 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:37:05.439438 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:37:05.441765 (+ 2327us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.14.254:45895: Server oryx/127.21.14.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":79,"threads_started":1}
W20260806 11:37:05.442027 31351 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:37:05.435395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:33519 (local address 127.21.14.254:45895)
0806 11:37:05.435586 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 11:37:05.435590 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:37:05.435718 (+ 128us) server_negotiation.cc:407] Connection header received
0806 11:37:05.435808 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:37:05.435811 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:37:05.435863 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:37:05.435974 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:37:05.436463 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:05.437189 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:05.438102 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:05.438383 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:05.438480 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:05.441912 (+ 3432us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.14.194:33519: BlockingRecv error: recv got EOF from 127.21.14.194:33519 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 06 11:37:05 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.195@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 11:37:05 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.195@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 11:37:05.675459 31353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:37:05.668976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:45895 (local address 127.21.14.195:36873)
0806 11:37:05.669385 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0806 11:37:05.669403 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 11:37:05.669562 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:37:05.669856 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:37:05.669859 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:37:05.670104 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:37:05.670226 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:05.670233 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:05.671190 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:05.671193 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:05.671761 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:05.671769 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:05.672209 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:05.672211 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:05.672275 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:05.672817 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:37:05.672829 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:37:05.675291 (+ 2462us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.14.254:45895: Server oryx/127.21.14.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":99,"threads_started":1}
W20260806 11:37:05.675697 31351 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:37:05.669138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:36873 (local address 127.21.14.254:45895)
0806 11:37:05.669350 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0806 11:37:05.669355 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:37:05.669493 (+ 138us) server_negotiation.cc:407] Connection header received
0806 11:37:05.669680 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:37:05.669683 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:37:05.669744 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:37:05.669831 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:37:05.670436 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:05.671073 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:05.671872 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:05.672095 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:05.672185 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:05.675533 (+ 3348us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.14.195:36873: BlockingRecv error: recv got EOF from 127.21.14.195:36873 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
Aug 06 11:37:06 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.193@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 11:37:06 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.193@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 11:37:06.217772 31354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:37:06.205079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:45895 (local address 127.21.14.193:36335)
0806 11:37:06.205415 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0806 11:37:06.205429 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 11:37:06.205603 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:37:06.205989 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:37:06.205991 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:37:06.206196 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:37:06.206315 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:06.206322 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:06.207349 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:06.207352 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:06.208288 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:06.208302 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:06.208927 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:06.208931 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:06.209027 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:06.215120 (+ 6093us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:37:06.215151 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:37:06.217588 (+ 2437us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.14.254:45895: Server oryx/127.21.14.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":125,"threads_started":1}
W20260806 11:37:06.217907 31355 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:37:06.205260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:36335 (local address 127.21.14.254:45895)
0806 11:37:06.205513 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0806 11:37:06.205520 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 11:37:06.205541 (+ 21us) server_negotiation.cc:407] Connection header received
0806 11:37:06.205716 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:37:06.205719 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:37:06.205768 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:37:06.205869 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:37:06.206562 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:06.207167 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:06.208432 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:06.208790 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:06.208918 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:06.217767 (+ 8849us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.14.193:36335: BlockingRecv error: recv got EOF from 127.21.14.193:36335 (error 108)
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":72,"threads_started":1}
Aug 06 11:37:06 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.194@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 11:37:06 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.194@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 11:37:06.449101 31356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:37:06.442934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:45895 (local address 127.21.14.194:36867)
0806 11:37:06.443302 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0806 11:37:06.443320 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 11:37:06.443444 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:37:06.443782 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:37:06.443785 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:37:06.443977 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:37:06.444087 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:06.444093 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:06.444976 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:06.444979 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:06.445574 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:06.445584 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:06.446131 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:06.446135 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:06.446226 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:06.446841 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:37:06.446920 (+ 79us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:37:06.448918 (+ 1998us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.14.254:45895: Server oryx/127.21.14.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":114,"threads_started":1}
W20260806 11:37:06.449206 31355 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:37:06.443033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:36867 (local address 127.21.14.254:45895)
0806 11:37:06.443194 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 11:37:06.443202 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 11:37:06.443406 (+ 204us) server_negotiation.cc:407] Connection header received
0806 11:37:06.443550 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:37:06.443553 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:37:06.443601 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:37:06.443676 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:37:06.444231 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:06.444789 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:06.445746 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:06.445987 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:06.446114 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:06.449082 (+ 2968us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.14.194:36867: BlockingRecv error: recv got EOF from 127.21.14.194:36867 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 06 11:37:06 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.195@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 11:37:06 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.195@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 11:37:06.682756 31357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:37:06.676537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:45895 (local address 127.21.14.195:48513)
0806 11:37:06.676848 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0806 11:37:06.676861 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 11:37:06.677012 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:37:06.677332 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:37:06.677336 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:37:06.677561 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:37:06.677663 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:06.677669 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:06.678500 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:06.678503 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:06.679145 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:06.679154 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:06.679659 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:06.679662 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:06.679736 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:06.680296 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:37:06.680309 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:37:06.682530 (+ 2221us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.14.254:45895: Server oryx/127.21.14.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":86,"threads_started":1}
W20260806 11:37:06.682873 31355 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:37:06.676622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:48513 (local address 127.21.14.254:45895)
0806 11:37:06.676782 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 11:37:06.676786 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:37:06.676946 (+ 160us) server_negotiation.cc:407] Connection header received
0806 11:37:06.677124 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:37:06.677128 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:37:06.677186 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:37:06.677278 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:37:06.677784 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:06.678377 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:06.679270 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:06.679504 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:06.679608 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:06.682702 (+ 3094us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.14.195:48513: BlockingRecv error: recv got EOF from 127.21.14.195:48513 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 06 11:37:07 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.193@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 11:37:07 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.193@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 11:37:07.225242 31358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:37:07.218961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:45895 (local address 127.21.14.193:47601)
0806 11:37:07.219305 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0806 11:37:07.219324 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 11:37:07.219433 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:37:07.219793 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:37:07.219798 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:37:07.220085 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:37:07.220257 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:07.220267 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:07.221140 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:07.221143 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:07.221761 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:07.221770 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:07.222325 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:07.222328 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:07.222436 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:07.223156 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:37:07.223169 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:37:07.225109 (+ 1940us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.14.254:45895: Server oryx/127.21.14.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":104,"threads_started":1}
W20260806 11:37:07.225565 31359 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:37:07.219067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:47601 (local address 127.21.14.254:45895)
0806 11:37:07.219368 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0806 11:37:07.219373 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:37:07.219417 (+ 44us) server_negotiation.cc:407] Connection header received
0806 11:37:07.219540 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:37:07.219545 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:37:07.219614 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:37:07.219689 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:37:07.220469 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:07.221017 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:07.221926 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:07.222173 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:07.222268 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:07.225376 (+ 3108us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.14.193:47601: BlockingRecv error: recv got EOF from 127.21.14.193:47601 (error 108)
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":80,"threads_started":1}
Aug 06 11:37:07 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.194@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 11:37:07 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.194@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 11:37:07.457288 31360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:37:07.450502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:45895 (local address 127.21.14.194:56875)
0806 11:37:07.451041 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0806 11:37:07.451055 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 11:37:07.451155 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:37:07.451456 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:37:07.451459 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:37:07.451702 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:37:07.451837 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:07.451846 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:07.452802 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:07.452805 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:07.453500 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:07.453510 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:07.454128 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:07.454131 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:07.454258 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:07.455026 (+ 768us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:37:07.455039 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:37:07.457091 (+ 2052us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.14.254:45895: Server oryx/127.21.14.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":100,"threads_started":1}
W20260806 11:37:07.457386 31359 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:37:07.450711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:56875 (local address 127.21.14.254:45895)
0806 11:37:07.450921 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0806 11:37:07.450924 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 11:37:07.451149 (+ 225us) server_negotiation.cc:407] Connection header received
0806 11:37:07.451227 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:37:07.451233 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:37:07.451297 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:37:07.451390 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:37:07.452025 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:07.452692 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:07.453651 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:07.453904 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:07.454063 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:07.457254 (+ 3191us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.14.194:56875: BlockingRecv error: recv got EOF from 127.21.14.194:56875 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 06 11:37:07 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.195@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 11:37:07 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.195@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 11:37:07.690271 31362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:37:07.683892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:45895 (local address 127.21.14.195:47109)
0806 11:37:07.684343 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0806 11:37:07.684364 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 11:37:07.684493 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:37:07.684813 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:37:07.684816 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:37:07.685030 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:37:07.685142 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:07.685149 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:07.686161 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:07.686165 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:07.686904 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:07.686921 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:07.687427 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:07.687431 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:07.687523 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:07.688098 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:37:07.688111 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:37:07.690069 (+ 1958us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.14.254:45895: Server oryx/127.21.14.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":104,"threads_started":1}
W20260806 11:37:07.690342 31359 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:37:07.684282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:47109 (local address 127.21.14.254:45895)
0806 11:37:07.684480 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 11:37:07.684484 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:37:07.684501 (+ 17us) server_negotiation.cc:407] Connection header received
0806 11:37:07.684593 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:37:07.684595 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:37:07.684647 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:37:07.684728 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:37:07.685304 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:07.686033 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:07.687038 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:07.687287 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:07.687418 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:07.690225 (+ 2807us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.14.195:47109: BlockingRecv error: recv got EOF from 127.21.14.195:47109 (error 108)
Metrics: {"server-negotiator.queue_time_us":108}
Aug 06 11:37:08 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.193@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 11:37:08 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.193@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 11:37:08.232414 31363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:37:08.226288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:45895 (local address 127.21.14.193:37859)
0806 11:37:08.226621 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0806 11:37:08.226635 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 11:37:08.226728 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:37:08.227121 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:37:08.227127 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:37:08.227341 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:37:08.227493 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:08.227501 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:08.228399 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:08.228402 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:08.228972 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:08.228981 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:08.229517 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:08.229520 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:08.229608 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:08.230245 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:37:08.230258 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:37:08.232275 (+ 2017us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.14.254:45895: Server oryx/127.21.14.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":130,"threads_started":1}
W20260806 11:37:08.232793 31365 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:37:08.226435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:37859 (local address 127.21.14.254:45895)
0806 11:37:08.226703 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0806 11:37:08.226708 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:37:08.226723 (+ 15us) server_negotiation.cc:407] Connection header received
0806 11:37:08.226832 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:37:08.226835 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:37:08.226925 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:37:08.227057 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:37:08.227617 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:08.228293 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:08.229111 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:08.229363 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:08.229457 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:08.232621 (+ 3164us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.14.193:37859: BlockingRecv error: recv got EOF from 127.21.14.193:37859 (error 108)
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":97,"threads_started":1}
Aug 06 11:37:08 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.194@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 11:37:08 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.194@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 11:37:08.464602 31366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:37:08.458296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:45895 (local address 127.21.14.194:39507)
0806 11:37:08.458622 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0806 11:37:08.458636 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 11:37:08.458743 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:37:08.459089 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:37:08.459092 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:37:08.459392 (+ 300us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:37:08.459509 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:08.459516 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:08.460444 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:08.460448 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:08.461077 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:08.461087 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:08.461573 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:08.461576 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:08.461663 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:08.462239 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:37:08.462251 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:37:08.464417 (+ 2166us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.14.254:45895: Server oryx/127.21.14.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":92,"threads_started":1}
W20260806 11:37:08.464720 31365 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:37:08.458484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:39507 (local address 127.21.14.254:45895)
0806 11:37:08.458671 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 11:37:08.458675 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:37:08.458695 (+ 20us) server_negotiation.cc:407] Connection header received
0806 11:37:08.458838 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:37:08.458840 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:37:08.458959 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:37:08.459061 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:37:08.459629 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:08.460333 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:08.461208 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:08.461454 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:08.461564 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:08.464601 (+ 3037us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.14.194:39507: BlockingRecv error: recv got EOF from 127.21.14.194:39507 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
Aug 06 11:37:08 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.195@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 11:37:08 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.195@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 11:37:08.700016 31367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:37:08.691426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:45895 (local address 127.21.14.195:57437)
0806 11:37:08.692061 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0806 11:37:08.692077 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 11:37:08.692201 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:37:08.692659 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:37:08.692662 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:37:08.692889 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:37:08.693160 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:08.693171 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:08.694595 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:08.694600 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:08.695804 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:08.695819 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:08.696355 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:08.696359 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:08.696439 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:08.697030 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:37:08.697045 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:37:08.699779 (+ 2734us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.14.254:45895: Server oryx/127.21.14.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":146,"threads_started":1}
W20260806 11:37:08.700244 31365 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:37:08.691617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:57437 (local address 127.21.14.254:45895)
0806 11:37:08.691763 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 11:37:08.691769 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 11:37:08.692255 (+ 486us) server_negotiation.cc:407] Connection header received
0806 11:37:08.692361 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:37:08.692364 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:37:08.692459 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:37:08.692582 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:37:08.693411 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:08.694428 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:08.695964 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:08.696220 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:08.696337 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:08.700055 (+ 3718us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.14.195:57437: BlockingRecv error: recv got EOF from 127.21.14.195:57437 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 06 11:37:09 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.193@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 11:37:09 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.193@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 11:37:09.239816 31368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:37:09.233432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:45895 (local address 127.21.14.193:58253)
0806 11:37:09.233811 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0806 11:37:09.233827 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 11:37:09.233968 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:37:09.234331 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:37:09.234335 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:37:09.234548 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:37:09.234659 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:09.234666 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:09.235553 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:09.235558 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:09.236213 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:09.236224 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:09.236743 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:09.236747 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:09.236839 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:09.237478 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:37:09.237489 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:37:09.239633 (+ 2144us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.14.254:45895: Server oryx/127.21.14.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":97,"threads_started":1}
W20260806 11:37:09.239943 31370 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:37:09.233552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:58253 (local address 127.21.14.254:45895)
0806 11:37:09.233842 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0806 11:37:09.233846 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:37:09.233931 (+ 85us) server_negotiation.cc:407] Connection header received
0806 11:37:09.234118 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:37:09.234122 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:37:09.234183 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:37:09.234269 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:37:09.234796 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:09.235406 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:09.236370 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:09.236619 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:09.236735 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:09.239790 (+ 3055us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.14.193:58253: BlockingRecv error: recv got EOF from 127.21.14.193:58253 (error 108)
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":98,"threads_started":1}
Aug 06 11:37:09 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.194@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 11:37:09 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.194@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 11:37:09.472918 31371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:37:09.465638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:45895 (local address 127.21.14.194:41999)
0806 11:37:09.465956 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0806 11:37:09.465969 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 11:37:09.466077 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:37:09.466438 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:37:09.466441 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:37:09.466643 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:37:09.466749 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:09.466755 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:09.467695 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:09.467698 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:09.468290 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:09.468300 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:09.469129 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:09.469136 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:09.469250 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:09.469994 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:37:09.470013 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:37:09.472778 (+ 2765us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.14.254:45895: Server oryx/127.21.14.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":126,"threads_started":1}
W20260806 11:37:09.473294 31370 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:37:09.465798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:41999 (local address 127.21.14.254:45895)
0806 11:37:09.465950 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 11:37:09.465954 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:37:09.466041 (+ 87us) server_negotiation.cc:407] Connection header received
0806 11:37:09.466182 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:37:09.466184 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:37:09.466232 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:37:09.466308 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:37:09.466989 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:09.467562 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:09.468446 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:09.468696 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:09.468824 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:09.473153 (+ 4329us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.14.194:41999: BlockingRecv error: recv got EOF from 127.21.14.194:41999 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 06 11:37:09 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.195@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 11:37:09 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.195@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 11:37:09.708417 31372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:37:09.701295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:45895 (local address 127.21.14.195:44159)
0806 11:37:09.701627 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0806 11:37:09.701650 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 11:37:09.701767 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:37:09.702181 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:37:09.702184 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:37:09.702374 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:37:09.702483 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:09.702490 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:09.703595 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:09.703599 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:09.704168 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:09.704176 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:09.705059 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:09.705064 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:09.705158 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:09.705920 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:37:09.705939 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:37:09.708272 (+ 2333us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.14.254:45895: Server oryx/127.21.14.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":153,"threads_started":1}
W20260806 11:37:09.708674 31370 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:37:09.701469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:44159 (local address 127.21.14.254:45895)
0806 11:37:09.701625 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 11:37:09.701629 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:37:09.701757 (+ 128us) server_negotiation.cc:407] Connection header received
0806 11:37:09.701850 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:37:09.701852 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:37:09.702014 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:37:09.702112 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:37:09.702663 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:09.703406 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:09.704338 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:09.704666 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:09.704816 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:09.708507 (+ 3691us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.14.195:44159: BlockingRecv error: recv got EOF from 127.21.14.195:44159 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 06 11:37:10 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.193@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 11:37:10 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.193@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 11:37:10.247507 31373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:37:10.241202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:45895 (local address 127.21.14.193:53199)
0806 11:37:10.241554 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0806 11:37:10.241573 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 11:37:10.241696 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:37:10.242015 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:37:10.242019 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:37:10.242247 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:37:10.242351 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:10.242357 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:10.243278 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:10.243281 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:10.244003 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:10.244021 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:10.244579 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:10.244583 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:10.244674 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:10.245225 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:37:10.245237 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:37:10.247335 (+ 2098us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.14.254:45895: Server oryx/127.21.14.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":88,"threads_started":1}
W20260806 11:37:10.247661 31374 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:37:10.241324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:53199 (local address 127.21.14.254:45895)
0806 11:37:10.241714 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0806 11:37:10.241717 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 11:37:10.241735 (+ 18us) server_negotiation.cc:407] Connection header received
0806 11:37:10.241805 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:37:10.241808 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:37:10.241854 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:37:10.241989 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:37:10.242482 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:10.243149 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:10.244178 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:10.244426 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:10.244576 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:10.247507 (+ 2931us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.14.193:53199: BlockingRecv error: recv got EOF from 127.21.14.193:53199 (error 108)
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":88,"threads_started":1}
Aug 06 11:37:10 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.194@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 11:37:10 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.194@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 11:37:10.480564 31375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:37:10.473972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:45895 (local address 127.21.14.194:33911)
0806 11:37:10.474454 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0806 11:37:10.474472 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 11:37:10.474658 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:37:10.474982 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:37:10.474985 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:37:10.475227 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:37:10.475327 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:10.475333 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:10.476139 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:10.476144 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:10.476731 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:10.476740 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:10.477248 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:10.477252 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:10.477343 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:10.477993 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:37:10.478098 (+ 105us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:37:10.480390 (+ 2292us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.14.254:45895: Server oryx/127.21.14.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":243,"threads_started":1}
W20260806 11:37:10.480687 31374 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:37:10.474285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:33911 (local address 127.21.14.254:45895)
0806 11:37:10.474473 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0806 11:37:10.474479 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 11:37:10.474589 (+ 110us) server_negotiation.cc:407] Connection header received
0806 11:37:10.474777 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:37:10.474781 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:37:10.474837 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:37:10.474955 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:37:10.475464 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:10.476017 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:10.476868 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:10.477113 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:10.477235 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:10.480573 (+ 3338us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.14.194:33911: BlockingRecv error: recv got EOF from 127.21.14.194:33911 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
Aug 06 11:37:10 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.195@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 11:37:10 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.195@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
W20260806 11:37:10.716085 31374 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:37:10.709551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:46369 (local address 127.21.14.254:45895)
0806 11:37:10.709712 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 11:37:10.709718 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 11:37:10.709912 (+ 194us) server_negotiation.cc:407] Connection header received
0806 11:37:10.710034 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:37:10.710037 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:37:10.710114 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:37:10.710218 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:37:10.710747 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:10.711371 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:10.712218 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:10.712465 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:10.712579 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:10.715930 (+ 3351us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.14.195:46369: BlockingRecv error: recv got EOF from 127.21.14.195:46369 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 11:37:10.716107 31376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:37:10.709426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:45895 (local address 127.21.14.195:46369)
0806 11:37:10.709783 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0806 11:37:10.709800 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 11:37:10.709938 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:37:10.710248 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:37:10.710251 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:37:10.710525 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:37:10.710623 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:10.710629 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:10.711489 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:10.711493 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:10.712091 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:10.712099 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:10.712589 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:10.712593 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:10.712679 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:10.713313 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:37:10.713325 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:37:10.715682 (+ 2357us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.14.254:45895: Server oryx/127.21.14.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":139,"threads_started":1}
Aug 06 11:37:11 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.193@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 11:37:11 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.193@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 11:37:11.258610 31377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:37:11.248355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:45895 (local address 127.21.14.193:35391)
0806 11:37:11.248701 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0806 11:37:11.248718 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 11:37:11.248828 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:37:11.249142 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:37:11.249144 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:37:11.249378 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:37:11.249502 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:11.249509 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:11.250346 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:11.250349 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:11.251056 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:11.251068 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:11.251603 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:11.251609 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:11.251715 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:11.255879 (+ 4164us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:37:11.255896 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:37:11.258408 (+ 2512us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.14.254:45895: Server oryx/127.21.14.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":133,"threads_started":1}
W20260806 11:37:11.258672 31378 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:37:11.248518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:35391 (local address 127.21.14.254:45895)
0806 11:37:11.248891 (+ 373us) server_negotiation.cc:206] Beginning negotiation
0806 11:37:11.248894 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 11:37:11.248908 (+ 14us) server_negotiation.cc:407] Connection header received
0806 11:37:11.248950 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:37:11.248952 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:37:11.248995 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:37:11.249096 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:37:11.249639 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:11.250218 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:11.251168 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:11.251421 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:11.251581 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:11.258546 (+ 6965us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.14.193:35391: BlockingRecv error: recv got EOF from 127.21.14.193:35391 (error 108)
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":88,"threads_started":1}
Aug 06 11:37:11 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.194@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 11:37:11 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.194@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 11:37:11.487820 31379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:37:11.481631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:45895 (local address 127.21.14.194:55073)
0806 11:37:11.481974 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0806 11:37:11.481989 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 11:37:11.482141 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:37:11.482545 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:37:11.482548 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:37:11.482839 (+ 291us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:37:11.483014 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:11.483022 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:11.483796 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:11.483799 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:11.484385 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:11.484394 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:11.485079 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:11.485083 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:11.485157 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:11.485685 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:37:11.485698 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:37:11.487680 (+ 1982us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.14.254:45895: Server oryx/127.21.14.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":99,"threads_started":1}
W20260806 11:37:11.488042 31378 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:37:11.481759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:55073 (local address 127.21.14.254:45895)
0806 11:37:11.481938 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 11:37:11.481943 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:37:11.482082 (+ 139us) server_negotiation.cc:407] Connection header received
0806 11:37:11.482261 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:37:11.482264 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:37:11.482346 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:37:11.482436 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:37:11.483131 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:11.483675 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:11.484525 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:11.484918 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:11.485020 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:11.487873 (+ 2853us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.14.194:55073: BlockingRecv error: recv got EOF from 127.21.14.194:55073 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 06 11:37:11 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.195@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 11:37:11 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.195@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 11:37:11.723061 31380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:37:11.717056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:45895 (local address 127.21.14.195:45701)
0806 11:37:11.717356 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0806 11:37:11.717370 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 11:37:11.717473 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:37:11.717707 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:37:11.717709 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:37:11.717904 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:37:11.718005 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:11.718010 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:11.718915 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:11.718918 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:11.719539 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:11.719549 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:11.720051 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:11.720053 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:11.720119 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:11.720710 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:37:11.720722 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:37:11.722914 (+ 2192us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.14.254:45895: Server oryx/127.21.14.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":106,"threads_started":1}
W20260806 11:37:11.723322 31378 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:37:11.717141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:45701 (local address 127.21.14.254:45895)
0806 11:37:11.717285 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 11:37:11.717291 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 11:37:11.717423 (+ 132us) server_negotiation.cc:407] Connection header received
0806 11:37:11.717542 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:37:11.717545 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:37:11.717597 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:37:11.717665 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:37:11.718167 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:11.718754 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:11.719671 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:11.719926 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:11.720014 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:11.723186 (+ 3172us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.14.195:45701: BlockingRecv error: recv got EOF from 127.21.14.195:45701 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 06 11:37:12 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.193@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 11:37:12 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.193@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 11:37:12.266050 31381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:37:12.259721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:45895 (local address 127.21.14.193:34035)
0806 11:37:12.260080 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0806 11:37:12.260100 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 11:37:12.260225 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:37:12.260541 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:37:12.260543 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:37:12.260752 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:37:12.260860 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:12.260866 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:12.261719 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:12.261722 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:12.262299 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:12.262308 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:12.262797 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:12.262800 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:12.262956 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:12.263611 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:37:12.263624 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:37:12.265838 (+ 2214us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.14.254:45895: Server oryx/127.21.14.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":101,"threads_started":1}
W20260806 11:37:12.266177 31382 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:37:12.259933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:34035 (local address 127.21.14.254:45895)
0806 11:37:12.260237 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0806 11:37:12.260240 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 11:37:12.260283 (+ 43us) server_negotiation.cc:407] Connection header received
0806 11:37:12.260340 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:37:12.260342 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:37:12.260385 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:37:12.260466 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:37:12.261018 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:12.261599 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:12.262423 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:12.262673 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:12.262761 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:12.265987 (+ 3226us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.14.193:34035: BlockingRecv error: recv got EOF from 127.21.14.193:34035 (error 108)
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":104,"threads_started":1}
Aug 06 11:37:12 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.194@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 11:37:12 dist-test-slave-7c35 krb5kdc[30228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.14.194@KRBTEST.COM for oryx/127.21.14.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 11:37:12.495533 31383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:37:12.489119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:45895 (local address 127.21.14.194:40227)
0806 11:37:12.489422 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0806 11:37:12.489436 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 11:37:12.489552 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:37:12.489880 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:37:12.489884 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:37:12.490142 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:37:12.490354 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:12.490362 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:12.491284 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:12.491287 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:12.491869 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:12.491880 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:12.492498 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:12.492501 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:12.492582 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:12.493186 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:37:12.493199 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:37:12.495352 (+ 2153us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.14.254:45895: Server oryx/127.21.14.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":100,"threads_started":1}
W20260806 11:37:12.495752 31382 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:37:12.489244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:40227 (local address 127.21.14.254:45895)
0806 11:37:12.489435 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 11:37:12.489439 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:37:12.489518 (+ 79us) server_negotiation.cc:407] Connection header received
0806 11:37:12.489648 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:37:12.489650 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:37:12.489704 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:37:12.489796 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:37:12.490529 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:12.491156 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:12.492059 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:12.492307 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:12.492405 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:12.495624 (+ 3219us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.14.194:40227: BlockingRecv error: recv got EOF from 127.21.14.194:40227 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 11:37:12.593456 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 30862
I20260806 11:37:12.602607 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 31026
I20260806 11:37:12.610211 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 31172
I20260806 11:37:12.619113 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 30785
2026-08-06T11:37:12Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18538 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-06T11:37:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T11:37:12Z Disabled control of system clock
I20260806 11:37:12.657047 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786016169045816-21563-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786016169045816-21563-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786016169045816-21563-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:41165
--webserver_interface=127.21.14.254
--webserver_port=0
--builtin_ntp_servers=127.21.14.212:39549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.14.254:41165
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 11:37:12.785125 31388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:37:12.785473 31388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:37:12.785512 31388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:37:12.790243 31388 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 11:37:12.790349 31388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:37:12.790366 31388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:37:12.790385 31388 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 11:37:12.790401 31388 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 11:37:12.795920 31388 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:39549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786016169045816-21563-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786016169045816-21563-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.14.254:41165
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:41165
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.14.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31388
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786016169045816-21563-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:37:12.797151 31388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:37:12.798661 31388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:37:12.809190 31393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:37:12.809190 31394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:37:12.809291 31388 server_base.cc:1034] running on GCE node
W20260806 11:37:12.809190 31396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:37:12.810252 31388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:37:12.811707 31388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:37:12.812945 31388 hybrid_clock.cc:648] HybridClock initialized: now 1786016232812910 us; error 62 us; skew 500 ppm
I20260806 11:37:12.815541 31388 webserver.cc:460] Webserver started at http://127.21.14.254:37169/ using document root <none> and password file <none>
I20260806 11:37:12.816262 31388 fs_manager.cc:359] Metadata directory not provided
I20260806 11:37:12.816375 31388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:37:12.816661 31388 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:37:12.819190 31388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786016169045816-21563-0/minicluster-data/master-0/data/instance:
uuid: "ab2dca0e8b3f4fa192c33659c02c3a8e"
format_stamp: "Formatted at 2026-08-06 11:37:12 on dist-test-slave-7c35"
server_key: "b7535a06ad1bead590b7d7d92c5d2246"
server_key_iv: "a523a074234a9a59e53193a80f4be226"
server_key_version: "encryptionkey@0"
I20260806 11:37:12.819880 31388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786016169045816-21563-0/minicluster-data/master-0/wal/instance:
uuid: "ab2dca0e8b3f4fa192c33659c02c3a8e"
format_stamp: "Formatted at 2026-08-06 11:37:12 on dist-test-slave-7c35"
server_key: "b7535a06ad1bead590b7d7d92c5d2246"
server_key_iv: "a523a074234a9a59e53193a80f4be226"
server_key_version: "encryptionkey@0"
I20260806 11:37:12.824354 31388 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 11:37:12.827633 31403 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:37:12.829320 31388 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 11:37:12.829556 31388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786016169045816-21563-0/minicluster-data/master-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786016169045816-21563-0/minicluster-data/master-0/wal
uuid: "ab2dca0e8b3f4fa192c33659c02c3a8e"
format_stamp: "Formatted at 2026-08-06 11:37:12 on dist-test-slave-7c35"
server_key: "b7535a06ad1bead590b7d7d92c5d2246"
server_key_iv: "a523a074234a9a59e53193a80f4be226"
server_key_version: "encryptionkey@0"
I20260806 11:37:12.829728 31388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786016169045816-21563-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786016169045816-21563-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786016169045816-21563-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:37:12.861264 31388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:37:12.862102 31388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:37:12.862340 31388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:37:12.870191 31388 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.254:41165
I20260806 11:37:12.870210 31463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.254:41165 every 8 connection(s)
I20260806 11:37:12.871351 31388 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
I20260806 11:37:12.875216 31464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:37:12.875856 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 31388
I20260806 11:37:12.875953 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786016169045816-21563-0/minicluster-data/master-0/wal/instance
I20260806 11:37:12.876343 21563 external_mini_cluster.cc:1428] Setting key 9d79702c8731c0ffba9dfdf30677086c
I20260806 11:37:12.882400 31464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab2dca0e8b3f4fa192c33659c02c3a8e: Bootstrap starting.
I20260806 11:37:12.885280 31464 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ab2dca0e8b3f4fa192c33659c02c3a8e: Neither blocks nor log segments found. Creating new log.
I20260806 11:37:12.886247 31464 log.cc:824] T 00000000000000000000000000000000 P ab2dca0e8b3f4fa192c33659c02c3a8e: Log is configured to *not* fsync() on all Append() calls
I20260806 11:37:12.887281 31467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:37:12.878016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34938 (local address 127.21.14.254:41165)
0806 11:37:12.878825 (+ 809us) server_negotiation.cc:206] Beginning negotiation
0806 11:37:12.878843 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 11:37:12.878965 (+ 122us) server_negotiation.cc:407] Connection header received
0806 11:37:12.879714 (+ 749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:37:12.879734 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:37:12.880184 (+ 450us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:37:12.880580 (+ 396us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:37:12.881054 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:12.882114 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:12.883239 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:12.883767 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:12.883970 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:12.885063 (+ 1093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:37:12.885069 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:37:12.885087 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 11:37:12.885631 (+ 544us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:37:12.885700 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 11:37:12.885724 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:37:12.885845 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 11:37:12.886016 (+ 171us) server_negotiation.cc:299] Negotiation successful
0806 11:37:12.886329 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":189,"threads_started":1}
I20260806 11:37:12.888904 31464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab2dca0e8b3f4fa192c33659c02c3a8e: No bootstrap required, opened a new log
I20260806 11:37:12.892958 31464 raft_consensus.cc:348] T 00000000000000000000000000000000 P ab2dca0e8b3f4fa192c33659c02c3a8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab2dca0e8b3f4fa192c33659c02c3a8e" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 41165 } }
I20260806 11:37:12.893268 31464 raft_consensus.cc:374] T 00000000000000000000000000000000 P ab2dca0e8b3f4fa192c33659c02c3a8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:37:12.893356 31464 raft_consensus.cc:729] T 00000000000000000000000000000000 P ab2dca0e8b3f4fa192c33659c02c3a8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab2dca0e8b3f4fa192c33659c02c3a8e, State: Initialized, Role: FOLLOWER
I20260806 11:37:12.893867 31464 consensus_queue.cc:260] T 00000000000000000000000000000000 P ab2dca0e8b3f4fa192c33659c02c3a8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab2dca0e8b3f4fa192c33659c02c3a8e" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 41165 } }
I20260806 11:37:12.894038 31464 raft_consensus.cc:388] T 00000000000000000000000000000000 P ab2dca0e8b3f4fa192c33659c02c3a8e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 11:37:12.894132 31464 raft_consensus.cc:482] T 00000000000000000000000000000000 P ab2dca0e8b3f4fa192c33659c02c3a8e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 11:37:12.894260 31464 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ab2dca0e8b3f4fa192c33659c02c3a8e [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:37:12.895591 31464 raft_consensus.cc:504] T 00000000000000000000000000000000 P ab2dca0e8b3f4fa192c33659c02c3a8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab2dca0e8b3f4fa192c33659c02c3a8e" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 41165 } }
I20260806 11:37:12.895975 31464 leader_election.cc:302] T 00000000000000000000000000000000 P ab2dca0e8b3f4fa192c33659c02c3a8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ab2dca0e8b3f4fa192c33659c02c3a8e; no voters:
I20260806 11:37:12.896325 31464 leader_election.cc:288] T 00000000000000000000000000000000 P ab2dca0e8b3f4fa192c33659c02c3a8e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 11:37:12.896487 31469 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ab2dca0e8b3f4fa192c33659c02c3a8e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:37:12.896772 31469 raft_consensus.cc:686] T 00000000000000000000000000000000 P ab2dca0e8b3f4fa192c33659c02c3a8e [term 1 LEADER]: Becoming Leader. State: Replica: ab2dca0e8b3f4fa192c33659c02c3a8e, State: Running, Role: LEADER
I20260806 11:37:12.897143 31469 consensus_queue.cc:237] T 00000000000000000000000000000000 P ab2dca0e8b3f4fa192c33659c02c3a8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab2dca0e8b3f4fa192c33659c02c3a8e" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 41165 } }
I20260806 11:37:12.897702 31464 sys_catalog.cc:564] T 00000000000000000000000000000000 P ab2dca0e8b3f4fa192c33659c02c3a8e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 11:37:12.898968 31471 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab2dca0e8b3f4fa192c33659c02c3a8e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ab2dca0e8b3f4fa192c33659c02c3a8e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab2dca0e8b3f4fa192c33659c02c3a8e" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 41165 } } }
I20260806 11:37:12.899117 31471 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab2dca0e8b3f4fa192c33659c02c3a8e [sys.catalog]: This master's current role is: LEADER
I20260806 11:37:12.899351 31472 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab2dca0e8b3f4fa192c33659c02c3a8e [sys.catalog]: SysCatalogTable state changed. Reason: New leader ab2dca0e8b3f4fa192c33659c02c3a8e. Latest consensus state: current_term: 1 leader_uuid: "ab2dca0e8b3f4fa192c33659c02c3a8e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab2dca0e8b3f4fa192c33659c02c3a8e" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 41165 } } }
I20260806 11:37:12.899521 31472 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab2dca0e8b3f4fa192c33659c02c3a8e [sys.catalog]: This master's current role is: LEADER
I20260806 11:37:12.899912 31479 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 11:37:12.903333 31479 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 11:37:12.909076 31479 catalog_manager.cc:1347] Generated new cluster ID: cc640790a1c44bb8a29e05f7c59267f3
I20260806 11:37:12.909186 31479 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 11:37:12.919059 31479 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 11:37:12.920078 31479 catalog_manager.cc:1504] Loading token signing keys...
I20260806 11:37:12.927654 31479 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ab2dca0e8b3f4fa192c33659c02c3a8e: Generated new TSK 0
I20260806 11:37:12.928504 31479 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 11:37:12.938524 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.193:0
--local_ip_for_outbound_sockets=127.21.14.193
--webserver_interface=127.21.14.193
--webserver_port=0
--tserver_master_addrs=127.21.14.254:41165
--builtin_ntp_servers=127.21.14.212:39549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260806 11:37:13.069298 31489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:37:13.069707 31489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:37:13.069757 31489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:37:13.074570 31489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:37:13.074687 31489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:37:13.074752 31489 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.193
I20260806 11:37:13.080569 31489 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:39549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.14.193
--webserver_port=0
--tserver_master_addrs=127.21.14.254:41165
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31489
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.193
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:37:13.081947 31489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:37:13.083629 31489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:37:13.096454 31497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:37:13.096462 31494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:37:13.096462 31495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:37:13.096812 31489 server_base.cc:1034] running on GCE node
I20260806 11:37:13.097383 31489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:37:13.098312 31489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:37:13.099594 31489 hybrid_clock.cc:648] HybridClock initialized: now 1786016233099565 us; error 48 us; skew 500 ppm
I20260806 11:37:13.102444 31489 webserver.cc:460] Webserver started at http://127.21.14.193:34069/ using document root <none> and password file <none>
I20260806 11:37:13.103307 31489 fs_manager.cc:359] Metadata directory not provided
I20260806 11:37:13.103401 31489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:37:13.103691 31489 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:37:13.106323 31489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-0/data/instance:
uuid: "e5a30adb8ba5417889042c372adac735"
format_stamp: "Formatted at 2026-08-06 11:37:13 on dist-test-slave-7c35"
server_key: "c89a8cd850b694ae02bd0f885d248eb9"
server_key_iv: "29d8f4d3b184a9118d669b3b33aad841"
server_key_version: "encryptionkey@0"
I20260806 11:37:13.107177 31489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-0/wal/instance:
uuid: "e5a30adb8ba5417889042c372adac735"
format_stamp: "Formatted at 2026-08-06 11:37:13 on dist-test-slave-7c35"
server_key: "c89a8cd850b694ae02bd0f885d248eb9"
server_key_iv: "29d8f4d3b184a9118d669b3b33aad841"
server_key_version: "encryptionkey@0"
I20260806 11:37:13.112367 31489 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260806 11:37:13.116349 31503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:37:13.118225 31489 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260806 11:37:13.118439 31489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-0/wal
uuid: "e5a30adb8ba5417889042c372adac735"
format_stamp: "Formatted at 2026-08-06 11:37:13 on dist-test-slave-7c35"
server_key: "c89a8cd850b694ae02bd0f885d248eb9"
server_key_iv: "29d8f4d3b184a9118d669b3b33aad841"
server_key_version: "encryptionkey@0"
I20260806 11:37:13.118630 31489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:37:13.145970 31489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:37:13.146989 31489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:37:13.147269 31489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:37:13.148017 31489 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:37:13.149227 31489 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:37:13.149313 31489 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:37:13.149395 31489 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:37:13.149442 31489 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:37:13.162317 31489 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.193:41827
I20260806 11:37:13.162381 31625 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.193:41827 every 8 connection(s)
I20260806 11:37:13.163753 31489 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
I20260806 11:37:13.167966 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 31489
I20260806 11:37:13.168094 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-0/wal/instance
I20260806 11:37:13.168376 21563 external_mini_cluster.cc:1428] Setting key e2b0a6f27a9cbe84289725a2770ea493
I20260806 11:37:13.171290 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.194:0
--local_ip_for_outbound_sockets=127.21.14.194
--webserver_interface=127.21.14.194
--webserver_port=0
--tserver_master_addrs=127.21.14.254:41165
--builtin_ntp_servers=127.21.14.212:39549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 11:37:13.178186 31467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:37:13.166298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:38823 (local address 127.21.14.254:41165)
0806 11:37:13.166492 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0806 11:37:13.166496 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:37:13.168099 (+ 1603us) server_negotiation.cc:407] Connection header received
0806 11:37:13.170411 (+ 2312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:37:13.170414 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:37:13.170511 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:37:13.170606 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:37:13.171869 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:13.172887 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:13.175446 (+ 2559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:13.175700 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:13.175831 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:13.177422 (+ 1591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:37:13.177425 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:37:13.177427 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 11:37:13.177444 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:37:13.177466 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 11:37:13.177475 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:37:13.177570 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 11:37:13.177780 (+ 210us) server_negotiation.cc:299] Negotiation successful
0806 11:37:13.177878 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260806 11:37:13.179500 31628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:37:13.166777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:41165 (local address 127.21.14.193:38823)
0806 11:37:13.167875 (+ 1098us) negotiation.cc:107] Waiting for socket to connect
0806 11:37:13.167925 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 11:37:13.170098 (+ 2173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:37:13.170833 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:37:13.170842 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:37:13.170973 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:37:13.171666 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:13.171686 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:13.173294 (+ 1608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:13.173297 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:13.175193 (+ 1896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:13.175214 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:13.175926 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:13.175933 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:13.176144 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:13.176917 (+ 773us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 11:37:13.176919 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:37:13.176972 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 11:37:13.177074 (+ 102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:37:13.177575 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:37:13.177590 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:37:13.177599 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 11:37:13.177758 (+ 159us) client_negotiation.cc:240] Negotiation successful
0806 11:37:13.178302 (+ 544us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":616,"thread_start_us":209,"threads_started":1}
I20260806 11:37:13.181782 31626 heartbeater.cc:347] Connected to a master server at 127.21.14.254:41165
I20260806 11:37:13.182228 31626 heartbeater.cc:464] Registering TS with master...
I20260806 11:37:13.183051 31626 heartbeater.cc:511] Master 127.21.14.254:41165 requested a full tablet report, sending...
I20260806 11:37:13.185264 31423 ts_manager.cc:194] Registered new tserver with Master: e5a30adb8ba5417889042c372adac735 (127.21.14.193:41827)
I20260806 11:37:13.187160 31423 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.14.193:38823
W20260806 11:37:13.305246 31630 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:37:13.305591 31630 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:37:13.305634 31630 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:37:13.310205 31630 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:37:13.310283 31630 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:37:13.310343 31630 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.194
I20260806 11:37:13.315673 31630 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:39549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.14.194
--webserver_port=0
--tserver_master_addrs=127.21.14.254:41165
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31630
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.194
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:37:13.316879 31630 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:37:13.318359 31630 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:37:13.329262 31638 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:37:13.329342 31636 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:37:13.329342 31635 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:37:13.329775 31630 server_base.cc:1034] running on GCE node
I20260806 11:37:13.330696 31630 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:37:13.331514 31630 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:37:13.333042 31630 hybrid_clock.cc:648] HybridClock initialized: now 1786016233332879 us; error 171 us; skew 500 ppm
I20260806 11:37:13.335697 31630 webserver.cc:460] Webserver started at http://127.21.14.194:36985/ using document root <none> and password file <none>
I20260806 11:37:13.336406 31630 fs_manager.cc:359] Metadata directory not provided
I20260806 11:37:13.336512 31630 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:37:13.336761 31630 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:37:13.339236 31630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-1/data/instance:
uuid: "e22ee332cffc435bbd5634ea38204655"
format_stamp: "Formatted at 2026-08-06 11:37:13 on dist-test-slave-7c35"
server_key: "9f939ae0a8abb238b814a114007755ae"
server_key_iv: "167cd7bff03d6b7732aded53ca82ac93"
server_key_version: "encryptionkey@0"
I20260806 11:37:13.339871 31630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-1/wal/instance:
uuid: "e22ee332cffc435bbd5634ea38204655"
format_stamp: "Formatted at 2026-08-06 11:37:13 on dist-test-slave-7c35"
server_key: "9f939ae0a8abb238b814a114007755ae"
server_key_iv: "167cd7bff03d6b7732aded53ca82ac93"
server_key_version: "encryptionkey@0"
I20260806 11:37:13.345166 31630 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 11:37:13.348886 31645 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:37:13.351020 31630 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 11:37:13.351213 31630 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-1/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-1/wal
uuid: "e22ee332cffc435bbd5634ea38204655"
format_stamp: "Formatted at 2026-08-06 11:37:13 on dist-test-slave-7c35"
server_key: "9f939ae0a8abb238b814a114007755ae"
server_key_iv: "167cd7bff03d6b7732aded53ca82ac93"
server_key_version: "encryptionkey@0"
I20260806 11:37:13.351374 31630 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:37:13.368366 31630 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:37:13.369280 31630 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:37:13.369549 31630 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:37:13.370278 31630 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:37:13.371507 31630 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:37:13.371584 31630 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:37:13.371668 31630 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:37:13.371722 31630 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:37:13.382087 31630 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.194:42275
I20260806 11:37:13.382113 31768 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.194:42275 every 8 connection(s)
I20260806 11:37:13.383132 31630 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
I20260806 11:37:13.391746 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 31630
I20260806 11:37:13.391883 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-1/wal/instance
I20260806 11:37:13.392215 21563 external_mini_cluster.cc:1428] Setting key b5b9b0ca82819812923e8b3e2a5d7f84
I20260806 11:37:13.394301 31467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:37:13.385241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:60883 (local address 127.21.14.254:41165)
0806 11:37:13.385399 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 11:37:13.385403 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:37:13.386707 (+ 1304us) server_negotiation.cc:407] Connection header received
0806 11:37:13.388279 (+ 1572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:37:13.388282 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:37:13.388334 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:37:13.388418 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:37:13.389396 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:13.390061 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:13.391452 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:13.391792 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:13.391931 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:13.393643 (+ 1712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:37:13.393646 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:37:13.393648 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 11:37:13.393661 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:37:13.393716 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 11:37:13.393729 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:37:13.393818 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 11:37:13.394081 (+ 263us) server_negotiation.cc:299] Negotiation successful
0806 11:37:13.394155 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 11:37:13.394759 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.195:0
--local_ip_for_outbound_sockets=127.21.14.195
--webserver_interface=127.21.14.195
--webserver_port=0
--tserver_master_addrs=127.21.14.254:41165
--builtin_ntp_servers=127.21.14.212:39549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 11:37:13.395474 31771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:37:13.385660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:41165 (local address 127.21.14.194:60883)
0806 11:37:13.386492 (+ 832us) negotiation.cc:107] Waiting for socket to connect
0806 11:37:13.386573 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0806 11:37:13.388026 (+ 1453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:37:13.388625 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:37:13.388633 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:37:13.388725 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:37:13.389255 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:13.389270 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:13.390212 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:13.390215 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:13.391288 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:13.391300 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:13.392139 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:13.392144 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:13.392390 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:13.393202 (+ 812us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 11:37:13.393204 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:37:13.393269 (+ 65us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 11:37:13.393437 (+ 168us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:37:13.393855 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:37:13.393868 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:37:13.393877 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 11:37:13.394048 (+ 171us) client_negotiation.cc:240] Negotiation successful
0806 11:37:13.394420 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":128,"threads_started":1}
I20260806 11:37:13.397468 31769 heartbeater.cc:347] Connected to a master server at 127.21.14.254:41165
I20260806 11:37:13.397780 31769 heartbeater.cc:464] Registering TS with master...
I20260806 11:37:13.398595 31769 heartbeater.cc:511] Master 127.21.14.254:41165 requested a full tablet report, sending...
I20260806 11:37:13.400064 31423 ts_manager.cc:194] Registered new tserver with Master: e22ee332cffc435bbd5634ea38204655 (127.21.14.194:42275)
I20260806 11:37:13.400712 31423 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.14.194:60883
W20260806 11:37:13.519379 31772 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:37:13.519711 31772 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:37:13.519749 31772 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:37:13.524292 31772 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:37:13.524372 31772 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:37:13.524432 31772 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.195
I20260806 11:37:13.529773 31772 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:39549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.14.195
--webserver_port=0
--tserver_master_addrs=127.21.14.254:41165
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31772
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.195
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:37:13.531224 31772 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:37:13.532722 31772 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:37:13.544137 31777 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:37:13.544124 31780 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:37:13.544124 31778 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:37:13.545415 31772 server_base.cc:1034] running on GCE node
I20260806 11:37:13.546145 31772 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:37:13.547472 31772 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:37:13.548781 31772 hybrid_clock.cc:648] HybridClock initialized: now 1786016233548749 us; error 42 us; skew 500 ppm
I20260806 11:37:13.551661 31772 webserver.cc:460] Webserver started at http://127.21.14.195:45969/ using document root <none> and password file <none>
I20260806 11:37:13.552444 31772 fs_manager.cc:359] Metadata directory not provided
I20260806 11:37:13.552561 31772 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:37:13.552847 31772 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:37:13.555339 31772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-2/data/instance:
uuid: "465c592d307a470cb62b290c8d17e7cc"
format_stamp: "Formatted at 2026-08-06 11:37:13 on dist-test-slave-7c35"
server_key: "0b91426a9ea80a2287c1605eb6439214"
server_key_iv: "e2e5e19d5fb102bcced760c54d6909b8"
server_key_version: "encryptionkey@0"
I20260806 11:37:13.556062 31772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-2/wal/instance:
uuid: "465c592d307a470cb62b290c8d17e7cc"
format_stamp: "Formatted at 2026-08-06 11:37:13 on dist-test-slave-7c35"
server_key: "0b91426a9ea80a2287c1605eb6439214"
server_key_iv: "e2e5e19d5fb102bcced760c54d6909b8"
server_key_version: "encryptionkey@0"
I20260806 11:37:13.560545 31772 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 11:37:13.563849 31786 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:37:13.565397 31772 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 11:37:13.565585 31772 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-2/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-2/wal
uuid: "465c592d307a470cb62b290c8d17e7cc"
format_stamp: "Formatted at 2026-08-06 11:37:13 on dist-test-slave-7c35"
server_key: "0b91426a9ea80a2287c1605eb6439214"
server_key_iv: "e2e5e19d5fb102bcced760c54d6909b8"
server_key_version: "encryptionkey@0"
I20260806 11:37:13.565744 31772 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:37:13.588346 31772 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:37:13.589205 31772 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:37:13.589437 31772 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:37:13.590397 31772 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:37:13.591576 31772 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:37:13.591655 31772 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:37:13.591734 31772 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:37:13.591786 31772 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:37:13.602263 31772 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.195:35591
I20260806 11:37:13.602349 31902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.195:35591 every 8 connection(s)
I20260806 11:37:13.603402 31772 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
I20260806 11:37:13.604288 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 31772
I20260806 11:37:13.604424 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-2/wal/instance
I20260806 11:37:13.604794 21563 external_mini_cluster.cc:1428] Setting key 21bb6840b4822008adeb4a749c69b83e
I20260806 11:37:13.614558 31467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:37:13.605843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:39367 (local address 127.21.14.254:41165)
0806 11:37:13.606025 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 11:37:13.606031 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 11:37:13.607632 (+ 1601us) server_negotiation.cc:407] Connection header received
0806 11:37:13.609131 (+ 1499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:37:13.609135 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:37:13.609195 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:37:13.609277 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:37:13.610296 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:13.610934 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:13.612289 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:13.612537 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:13.612659 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:13.613985 (+ 1326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:37:13.613989 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:37:13.613992 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 11:37:13.614010 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:37:13.614034 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 11:37:13.614044 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:37:13.614141 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 11:37:13.614310 (+ 169us) server_negotiation.cc:299] Negotiation successful
0806 11:37:13.614385 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 11:37:13.616171 31905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:37:13.606553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:41165 (local address 127.21.14.195:39367)
0806 11:37:13.607481 (+ 928us) negotiation.cc:107] Waiting for socket to connect
0806 11:37:13.607529 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 11:37:13.608892 (+ 1363us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:37:13.609468 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:37:13.609476 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:37:13.609572 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:37:13.610135 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:13.610149 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:13.611108 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:13.611111 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:13.612141 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:13.612153 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:13.612723 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:13.612728 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:13.612925 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:13.613658 (+ 733us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 11:37:13.613659 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:37:13.613709 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 11:37:13.613788 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:37:13.614143 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:37:13.614156 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:37:13.614167 (+ 11us) client_negotiation.cc:769] Sending connection context
0806 11:37:13.614310 (+ 143us) client_negotiation.cc:240] Negotiation successful
0806 11:37:13.614654 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":126,"threads_started":1}
I20260806 11:37:13.617867 31903 heartbeater.cc:347] Connected to a master server at 127.21.14.254:41165
I20260806 11:37:13.618208 31903 heartbeater.cc:464] Registering TS with master...
I20260806 11:37:13.619103 31903 heartbeater.cc:511] Master 127.21.14.254:41165 requested a full tablet report, sending...
I20260806 11:37:13.620584 31423 ts_manager.cc:194] Registered new tserver with Master: 465c592d307a470cb62b290c8d17e7cc (127.21.14.195:35591)
I20260806 11:37:13.621428 31423 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.14.195:39367
I20260806 11:37:13.630025 21563 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 11:37:13.633662 31467 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:37:13.632493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34946 (local address 127.21.14.254:41165)
0806 11:37:13.632688 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0806 11:37:13.632694 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 11:37:13.632847 (+ 153us) server_negotiation.cc:407] Connection header received
0806 11:37:13.633022 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:37:13.633026 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:37:13.633085 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:37:13.633186 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:37:13.633447 (+ 261us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34946: BlockingRecv error: recv got EOF from 127.0.0.1:34946 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 11:37:13.633817 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 31489
I20260806 11:37:13.645495 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 31630
I20260806 11:37:13.654431 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 31772
I20260806 11:37:13.663154 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 31388
2026-08-06T11:37:13Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (1041 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-06T11:37:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T11:37:13Z Disabled control of system clock
I20260806 11:37:13.698689 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786016169045816-21563-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786016169045816-21563-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786016169045816-21563-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:41705
--webserver_interface=127.21.14.254
--webserver_port=0
--builtin_ntp_servers=127.21.14.212:34155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.14.254:41705
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 11:37:13.827239 31915 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:37:13.827631 31915 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:37:13.827719 31915 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:37:13.832050 31915 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 11:37:13.832188 31915 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:37:13.832227 31915 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:37:13.832267 31915 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 11:37:13.832301 31915 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 11:37:13.837998 31915 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:34155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786016169045816-21563-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786016169045816-21563-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.14.254:41705
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:41705
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.14.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31915
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786016169045816-21563-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:37:13.839438 31915 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:37:13.841070 31915 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:37:13.850893 31920 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:37:13.850886 31924 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:37:13.851055 31915 server_base.cc:1034] running on GCE node
W20260806 11:37:13.850886 31921 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:37:13.851919 31915 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:37:13.853164 31915 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:37:13.854346 31915 hybrid_clock.cc:648] HybridClock initialized: now 1786016233854311 us; error 60 us; skew 500 ppm
I20260806 11:37:13.856531 31915 webserver.cc:460] Webserver started at http://127.21.14.254:40027/ using document root <none> and password file <none>
I20260806 11:37:13.857146 31915 fs_manager.cc:359] Metadata directory not provided
I20260806 11:37:13.857210 31915 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:37:13.857388 31915 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:37:13.859635 31915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786016169045816-21563-0/minicluster-data/master-0/data/instance:
uuid: "f2efd11d890d4603a8de9a05f2a61b8e"
format_stamp: "Formatted at 2026-08-06 11:37:13 on dist-test-slave-7c35"
server_key: "0b0e06d2c43857d012b85a5d60e8a8ca"
server_key_iv: "118a833186334a50e814309301eaaf3e"
server_key_version: "encryptionkey@0"
I20260806 11:37:13.860191 31915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786016169045816-21563-0/minicluster-data/master-0/wal/instance:
uuid: "f2efd11d890d4603a8de9a05f2a61b8e"
format_stamp: "Formatted at 2026-08-06 11:37:13 on dist-test-slave-7c35"
server_key: "0b0e06d2c43857d012b85a5d60e8a8ca"
server_key_iv: "118a833186334a50e814309301eaaf3e"
server_key_version: "encryptionkey@0"
I20260806 11:37:13.864292 31915 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 11:37:13.867257 31930 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:37:13.868676 31915 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 11:37:13.868803 31915 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786016169045816-21563-0/minicluster-data/master-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786016169045816-21563-0/minicluster-data/master-0/wal
uuid: "f2efd11d890d4603a8de9a05f2a61b8e"
format_stamp: "Formatted at 2026-08-06 11:37:13 on dist-test-slave-7c35"
server_key: "0b0e06d2c43857d012b85a5d60e8a8ca"
server_key_iv: "118a833186334a50e814309301eaaf3e"
server_key_version: "encryptionkey@0"
I20260806 11:37:13.868921 31915 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786016169045816-21563-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786016169045816-21563-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786016169045816-21563-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:37:13.873862 31915 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:37:13.874518 31915 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:37:13.874675 31915 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:37:13.882165 31915 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.254:41705
I20260806 11:37:13.882196 31987 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.254:41705 every 8 connection(s)
I20260806 11:37:13.883289 31915 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
I20260806 11:37:13.886426 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 31915
I20260806 11:37:13.886595 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786016169045816-21563-0/minicluster-data/master-0/wal/instance
I20260806 11:37:13.887014 21563 external_mini_cluster.cc:1428] Setting key 21242cf8ee127dfa389270774ac282e0
I20260806 11:37:13.887300 31988 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:37:13.893733 31991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:37:13.888792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48942 (local address 127.21.14.254:41705)
0806 11:37:13.889592 (+ 800us) server_negotiation.cc:206] Beginning negotiation
0806 11:37:13.889609 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 11:37:13.889650 (+ 41us) server_negotiation.cc:407] Connection header received
0806 11:37:13.890371 (+ 721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:37:13.890391 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:37:13.890971 (+ 580us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:37:13.891375 (+ 404us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:37:13.892279 (+ 904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:37:13.892284 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:37:13.892297 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 11:37:13.892310 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:37:13.892381 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 11:37:13.892399 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:37:13.892585 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0806 11:37:13.892665 (+ 80us) server_negotiation.cc:299] Negotiation successful
0806 11:37:13.892940 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":162,"threads_started":1}
I20260806 11:37:13.894831 31988 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2efd11d890d4603a8de9a05f2a61b8e: Bootstrap starting.
I20260806 11:37:13.897516 31988 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f2efd11d890d4603a8de9a05f2a61b8e: Neither blocks nor log segments found. Creating new log.
I20260806 11:37:13.898279 31988 log.cc:824] T 00000000000000000000000000000000 P f2efd11d890d4603a8de9a05f2a61b8e: Log is configured to *not* fsync() on all Append() calls
I20260806 11:37:13.900533 31988 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2efd11d890d4603a8de9a05f2a61b8e: No bootstrap required, opened a new log
I20260806 11:37:13.905211 31988 raft_consensus.cc:348] T 00000000000000000000000000000000 P f2efd11d890d4603a8de9a05f2a61b8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2efd11d890d4603a8de9a05f2a61b8e" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 41705 } }
I20260806 11:37:13.905512 31988 raft_consensus.cc:374] T 00000000000000000000000000000000 P f2efd11d890d4603a8de9a05f2a61b8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:37:13.905560 31988 raft_consensus.cc:729] T 00000000000000000000000000000000 P f2efd11d890d4603a8de9a05f2a61b8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2efd11d890d4603a8de9a05f2a61b8e, State: Initialized, Role: FOLLOWER
I20260806 11:37:13.905993 31988 consensus_queue.cc:260] T 00000000000000000000000000000000 P f2efd11d890d4603a8de9a05f2a61b8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2efd11d890d4603a8de9a05f2a61b8e" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 41705 } }
I20260806 11:37:13.906112 31988 raft_consensus.cc:388] T 00000000000000000000000000000000 P f2efd11d890d4603a8de9a05f2a61b8e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 11:37:13.906155 31988 raft_consensus.cc:482] T 00000000000000000000000000000000 P f2efd11d890d4603a8de9a05f2a61b8e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 11:37:13.906234 31988 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f2efd11d890d4603a8de9a05f2a61b8e [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:37:13.907397 31988 raft_consensus.cc:504] T 00000000000000000000000000000000 P f2efd11d890d4603a8de9a05f2a61b8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2efd11d890d4603a8de9a05f2a61b8e" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 41705 } }
I20260806 11:37:13.907760 31988 leader_election.cc:302] T 00000000000000000000000000000000 P f2efd11d890d4603a8de9a05f2a61b8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f2efd11d890d4603a8de9a05f2a61b8e; no voters:
I20260806 11:37:13.908113 31988 leader_election.cc:288] T 00000000000000000000000000000000 P f2efd11d890d4603a8de9a05f2a61b8e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 11:37:13.908262 31993 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f2efd11d890d4603a8de9a05f2a61b8e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:37:13.908548 31993 raft_consensus.cc:686] T 00000000000000000000000000000000 P f2efd11d890d4603a8de9a05f2a61b8e [term 1 LEADER]: Becoming Leader. State: Replica: f2efd11d890d4603a8de9a05f2a61b8e, State: Running, Role: LEADER
I20260806 11:37:13.908892 31993 consensus_queue.cc:237] T 00000000000000000000000000000000 P f2efd11d890d4603a8de9a05f2a61b8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2efd11d890d4603a8de9a05f2a61b8e" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 41705 } }
I20260806 11:37:13.909577 31988 sys_catalog.cc:564] T 00000000000000000000000000000000 P f2efd11d890d4603a8de9a05f2a61b8e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 11:37:13.910579 31995 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2efd11d890d4603a8de9a05f2a61b8e [sys.catalog]: SysCatalogTable state changed. Reason: New leader f2efd11d890d4603a8de9a05f2a61b8e. Latest consensus state: current_term: 1 leader_uuid: "f2efd11d890d4603a8de9a05f2a61b8e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2efd11d890d4603a8de9a05f2a61b8e" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 41705 } } }
I20260806 11:37:13.910750 31995 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2efd11d890d4603a8de9a05f2a61b8e [sys.catalog]: This master's current role is: LEADER
I20260806 11:37:13.910764 31994 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2efd11d890d4603a8de9a05f2a61b8e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f2efd11d890d4603a8de9a05f2a61b8e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2efd11d890d4603a8de9a05f2a61b8e" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 41705 } } }
I20260806 11:37:13.910835 31994 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2efd11d890d4603a8de9a05f2a61b8e [sys.catalog]: This master's current role is: LEADER
I20260806 11:37:13.911348 32002 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 11:37:13.914836 32002 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 11:37:13.921406 32002 catalog_manager.cc:1347] Generated new cluster ID: 408a6016cd3c468186d0a2634fbbea53
I20260806 11:37:13.921499 32002 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 11:37:13.936322 32002 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 11:37:13.936729 32002 catalog_manager.cc:1504] Loading token signing keys...
I20260806 11:37:13.944178 32002 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f2efd11d890d4603a8de9a05f2a61b8e: Generated new TSK 0
I20260806 11:37:13.945176 32002 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 11:37:13.961342 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.193:0
--local_ip_for_outbound_sockets=127.21.14.193
--webserver_interface=127.21.14.193
--webserver_port=0
--tserver_master_addrs=127.21.14.254:41705
--builtin_ntp_servers=127.21.14.212:34155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 11:37:14.088124 32013 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:37:14.088452 32013 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:37:14.088506 32013 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:37:14.093294 32013 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:37:14.093403 32013 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:37:14.093473 32013 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.193
I20260806 11:37:14.099468 32013 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:34155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.14.193
--webserver_port=0
--tserver_master_addrs=127.21.14.254:41705
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32013
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.193
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:37:14.100705 32013 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:37:14.102211 32013 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:37:14.114058 32021 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:37:14.114372 32019 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:37:14.114316 32013 server_base.cc:1034] running on GCE node
W20260806 11:37:14.114146 32018 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:37:14.115267 32013 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:37:14.116094 32013 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:37:14.117482 32013 hybrid_clock.cc:648] HybridClock initialized: now 1786016234117433 us; error 65 us; skew 500 ppm
I20260806 11:37:14.120007 32013 webserver.cc:460] Webserver started at http://127.21.14.193:45379/ using document root <none> and password file <none>
I20260806 11:37:14.120713 32013 fs_manager.cc:359] Metadata directory not provided
I20260806 11:37:14.120906 32013 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:37:14.121207 32013 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:37:14.123821 32013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-0/data/instance:
uuid: "5de59bc9dd9142ce9b12541f03442546"
format_stamp: "Formatted at 2026-08-06 11:37:14 on dist-test-slave-7c35"
server_key: "eb4a904b42ef0ac99a8cec741956ba10"
server_key_iv: "68b69acf304f14885c2707b289ab3810"
server_key_version: "encryptionkey@0"
I20260806 11:37:14.124519 32013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-0/wal/instance:
uuid: "5de59bc9dd9142ce9b12541f03442546"
format_stamp: "Formatted at 2026-08-06 11:37:14 on dist-test-slave-7c35"
server_key: "eb4a904b42ef0ac99a8cec741956ba10"
server_key_iv: "68b69acf304f14885c2707b289ab3810"
server_key_version: "encryptionkey@0"
I20260806 11:37:14.129861 32013 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 11:37:14.133236 32027 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:37:14.135107 32013 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 11:37:14.135299 32013 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-0/wal
uuid: "5de59bc9dd9142ce9b12541f03442546"
format_stamp: "Formatted at 2026-08-06 11:37:14 on dist-test-slave-7c35"
server_key: "eb4a904b42ef0ac99a8cec741956ba10"
server_key_iv: "68b69acf304f14885c2707b289ab3810"
server_key_version: "encryptionkey@0"
I20260806 11:37:14.135473 32013 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:37:14.141424 32013 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:37:14.142421 32013 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:37:14.142684 32013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:37:14.143453 32013 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:37:14.144620 32013 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:37:14.144699 32013 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:37:14.144780 32013 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:37:14.144827 32013 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:37:14.156133 32013 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.193:40081
I20260806 11:37:14.156158 32149 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.193:40081 every 8 connection(s)
I20260806 11:37:14.157562 32013 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
I20260806 11:37:14.160223 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 32013
I20260806 11:37:14.160372 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-0/wal/instance
I20260806 11:37:14.160889 21563 external_mini_cluster.cc:1428] Setting key c160ba6168c520e3b0a6c65e337c903a
I20260806 11:37:14.163561 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.194:0
--local_ip_for_outbound_sockets=127.21.14.194
--webserver_interface=127.21.14.194
--webserver_port=0
--tserver_master_addrs=127.21.14.254:41705
--builtin_ntp_servers=127.21.14.212:34155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260806 11:37:14.165824 31991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:37:14.159912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:41823 (local address 127.21.14.254:41705)
0806 11:37:14.161461 (+ 1549us) server_negotiation.cc:206] Beginning negotiation
0806 11:37:14.161467 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 11:37:14.161728 (+ 261us) server_negotiation.cc:407] Connection header received
0806 11:37:14.163729 (+ 2001us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:37:14.163735 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:37:14.163787 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:37:14.163884 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:37:14.165284 (+ 1400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:37:14.165288 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:37:14.165291 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 11:37:14.165306 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:37:14.165333 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 11:37:14.165348 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:37:14.165456 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 11:37:14.165619 (+ 163us) server_negotiation.cc:299] Negotiation successful
0806 11:37:14.165674 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1432}
I20260806 11:37:14.166997 32152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:37:14.160462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:41705 (local address 127.21.14.193:41823)
0806 11:37:14.161543 (+ 1081us) negotiation.cc:107] Waiting for socket to connect
0806 11:37:14.161608 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0806 11:37:14.163448 (+ 1840us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:37:14.164109 (+ 661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:37:14.164118 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:37:14.164230 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:37:14.164955 (+ 725us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 11:37:14.164957 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:37:14.165009 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 11:37:14.165102 (+ 93us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:37:14.165446 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:37:14.165455 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:37:14.165464 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 11:37:14.165678 (+ 214us) client_negotiation.cc:240] Negotiation successful
0806 11:37:14.165987 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":175,"threads_started":1}
I20260806 11:37:14.168622 32150 heartbeater.cc:347] Connected to a master server at 127.21.14.254:41705
I20260806 11:37:14.168957 32150 heartbeater.cc:464] Registering TS with master...
I20260806 11:37:14.169605 32150 heartbeater.cc:511] Master 127.21.14.254:41705 requested a full tablet report, sending...
I20260806 11:37:14.172515 31949 ts_manager.cc:194] Registered new tserver with Master: 5de59bc9dd9142ce9b12541f03442546 (127.21.14.193:40081)
W20260806 11:37:14.297392 32153 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:37:14.297706 32153 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:37:14.297750 32153 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:37:14.302011 32153 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:37:14.302088 32153 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:37:14.302151 32153 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.194
I20260806 11:37:14.307224 32153 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:34155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.14.194
--webserver_port=0
--tserver_master_addrs=127.21.14.254:41705
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32153
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.194
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:37:14.308408 32153 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:37:14.310184 32153 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:37:14.322352 32163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:37:14.322314 32161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:37:14.322336 32160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:37:14.322721 32153 server_base.cc:1034] running on GCE node
I20260806 11:37:14.323374 32153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:37:14.324280 32153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:37:14.325525 32153 hybrid_clock.cc:648] HybridClock initialized: now 1786016234325476 us; error 63 us; skew 500 ppm
I20260806 11:37:14.328003 32153 webserver.cc:460] Webserver started at http://127.21.14.194:39435/ using document root <none> and password file <none>
I20260806 11:37:14.328686 32153 fs_manager.cc:359] Metadata directory not provided
I20260806 11:37:14.328794 32153 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:37:14.329058 32153 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:37:14.331493 32153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-1/data/instance:
uuid: "f300e303d02445c2902b1619a6b493fe"
format_stamp: "Formatted at 2026-08-06 11:37:14 on dist-test-slave-7c35"
server_key: "c3de60264c4a30912a4fc72d1e413cf8"
server_key_iv: "5e4407e6ce8f8ae5360298beba6274d4"
server_key_version: "encryptionkey@0"
I20260806 11:37:14.332156 32153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-1/wal/instance:
uuid: "f300e303d02445c2902b1619a6b493fe"
format_stamp: "Formatted at 2026-08-06 11:37:14 on dist-test-slave-7c35"
server_key: "c3de60264c4a30912a4fc72d1e413cf8"
server_key_iv: "5e4407e6ce8f8ae5360298beba6274d4"
server_key_version: "encryptionkey@0"
I20260806 11:37:14.336926 32153 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260806 11:37:14.340390 32169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:37:14.341986 32153 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 11:37:14.342175 32153 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-1/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-1/wal
uuid: "f300e303d02445c2902b1619a6b493fe"
format_stamp: "Formatted at 2026-08-06 11:37:14 on dist-test-slave-7c35"
server_key: "c3de60264c4a30912a4fc72d1e413cf8"
server_key_iv: "5e4407e6ce8f8ae5360298beba6274d4"
server_key_version: "encryptionkey@0"
I20260806 11:37:14.342339 32153 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:37:14.347832 32153 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:37:14.348603 32153 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:37:14.348825 32153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:37:14.349462 32153 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:37:14.350528 32153 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:37:14.350600 32153 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:37:14.350680 32153 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:37:14.350735 32153 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:37:14.360458 32153 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.194:32951
I20260806 11:37:14.360476 32292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.194:32951 every 8 connection(s)
I20260806 11:37:14.361516 32153 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
I20260806 11:37:14.362461 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 32153
I20260806 11:37:14.362560 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-1/wal/instance
I20260806 11:37:14.362937 21563 external_mini_cluster.cc:1428] Setting key e9f44a0c66601abb0065ed07346b16d2
I20260806 11:37:14.365298 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.195:0
--local_ip_for_outbound_sockets=127.21.14.195
--webserver_interface=127.21.14.195
--webserver_port=0
--tserver_master_addrs=127.21.14.254:41705
--builtin_ntp_servers=127.21.14.212:34155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260806 11:37:14.368534 31991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:37:14.363678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:50571 (local address 127.21.14.254:41705)
0806 11:37:14.363853 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 11:37:14.363858 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:37:14.365143 (+ 1285us) server_negotiation.cc:407] Connection header received
0806 11:37:14.366590 (+ 1447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:37:14.366593 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:37:14.366641 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:37:14.366721 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:37:14.368025 (+ 1304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:37:14.368029 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:37:14.368031 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 11:37:14.368047 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:37:14.368075 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 11:37:14.368089 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:37:14.368192 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 11:37:14.368337 (+ 145us) server_negotiation.cc:299] Negotiation successful
0806 11:37:14.368389 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 11:37:14.369493 32295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:37:14.364207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:41705 (local address 127.21.14.194:50571)
0806 11:37:14.364988 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0806 11:37:14.365034 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 11:37:14.366361 (+ 1327us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:37:14.366966 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:37:14.366974 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:37:14.367085 (+ 111us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:37:14.367734 (+ 649us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 11:37:14.367735 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:37:14.367782 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 11:37:14.367861 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:37:14.368174 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:37:14.368183 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:37:14.368189 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 11:37:14.368390 (+ 201us) client_negotiation.cc:240] Negotiation successful
0806 11:37:14.368667 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":134,"threads_started":1}
I20260806 11:37:14.370958 32293 heartbeater.cc:347] Connected to a master server at 127.21.14.254:41705
I20260806 11:37:14.371235 32293 heartbeater.cc:464] Registering TS with master...
I20260806 11:37:14.371712 32293 heartbeater.cc:511] Master 127.21.14.254:41705 requested a full tablet report, sending...
I20260806 11:37:14.373126 31949 ts_manager.cc:194] Registered new tserver with Master: f300e303d02445c2902b1619a6b493fe (127.21.14.194:32951)
W20260806 11:37:14.493072 32297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:37:14.493427 32297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:37:14.493474 32297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:37:14.497720 32297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:37:14.497795 32297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:37:14.497861 32297 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.195
I20260806 11:37:14.502872 32297 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:34155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.14.195
--webserver_port=0
--tserver_master_addrs=127.21.14.254:41705
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32297
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.195
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:37:14.504065 32297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:37:14.505501 32297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:37:14.516966 32303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:37:14.516965 32305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:37:14.516959 32302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:37:14.517513 32297 server_base.cc:1034] running on GCE node
I20260806 11:37:14.518515 32297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:37:14.519306 32297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:37:14.520545 32297 hybrid_clock.cc:648] HybridClock initialized: now 1786016234520479 us; error 53 us; skew 500 ppm
I20260806 11:37:14.523495 32297 webserver.cc:460] Webserver started at http://127.21.14.195:38621/ using document root <none> and password file <none>
I20260806 11:37:14.524590 32297 fs_manager.cc:359] Metadata directory not provided
I20260806 11:37:14.524682 32297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:37:14.524885 32297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:37:14.527422 32297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-2/data/instance:
uuid: "d8d122f3470d4534986c6489c157b0f6"
format_stamp: "Formatted at 2026-08-06 11:37:14 on dist-test-slave-7c35"
server_key: "2506c0f95053ad39608e6e34957b2201"
server_key_iv: "5cc8d7eb237c3d02bd32af6885492fc8"
server_key_version: "encryptionkey@0"
I20260806 11:37:14.528056 32297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-2/wal/instance:
uuid: "d8d122f3470d4534986c6489c157b0f6"
format_stamp: "Formatted at 2026-08-06 11:37:14 on dist-test-slave-7c35"
server_key: "2506c0f95053ad39608e6e34957b2201"
server_key_iv: "5cc8d7eb237c3d02bd32af6885492fc8"
server_key_version: "encryptionkey@0"
I20260806 11:37:14.533133 32297 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 11:37:14.536469 32311 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:37:14.538424 32297 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 11:37:14.538636 32297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-2/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-2/wal
uuid: "d8d122f3470d4534986c6489c157b0f6"
format_stamp: "Formatted at 2026-08-06 11:37:14 on dist-test-slave-7c35"
server_key: "2506c0f95053ad39608e6e34957b2201"
server_key_iv: "5cc8d7eb237c3d02bd32af6885492fc8"
server_key_version: "encryptionkey@0"
I20260806 11:37:14.538817 32297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:37:14.545099 32297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:37:14.545976 32297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:37:14.546253 32297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:37:14.547003 32297 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:37:14.548178 32297 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:37:14.548231 32297 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:37:14.548278 32297 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:37:14.548333 32297 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:37:14.558813 32297 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.195:41369
I20260806 11:37:14.558838 32434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.195:41369 every 8 connection(s)
I20260806 11:37:14.560031 32297 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
I20260806 11:37:14.563817 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 32297
I20260806 11:37:14.563949 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786016169045816-21563-0/minicluster-data/ts-2/wal/instance
I20260806 11:37:14.564195 21563 external_mini_cluster.cc:1428] Setting key 0f2cead37a7987134aa4441ebf51082b
I20260806 11:37:14.568145 31991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:37:14.562371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:50155 (local address 127.21.14.254:41705)
0806 11:37:14.562571 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0806 11:37:14.562576 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:37:14.563907 (+ 1331us) server_negotiation.cc:407] Connection header received
0806 11:37:14.565565 (+ 1658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:37:14.565570 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:37:14.565617 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:37:14.565715 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:37:14.567668 (+ 1953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:37:14.567672 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:37:14.567674 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 11:37:14.567688 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:37:14.567712 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 11:37:14.567726 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:37:14.567817 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 11:37:14.567963 (+ 146us) server_negotiation.cc:299] Negotiation successful
0806 11:37:14.568007 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260806 11:37:14.569240 32438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:37:14.562841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:41705 (local address 127.21.14.195:50155)
0806 11:37:14.563719 (+ 878us) negotiation.cc:107] Waiting for socket to connect
0806 11:37:14.563767 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 11:37:14.565285 (+ 1518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:37:14.565949 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:37:14.565959 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:37:14.566113 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:37:14.567291 (+ 1178us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 11:37:14.567293 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:37:14.567366 (+ 73us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 11:37:14.567462 (+ 96us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:37:14.567818 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:37:14.567825 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:37:14.567838 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 11:37:14.567994 (+ 156us) client_negotiation.cc:240] Negotiation successful
0806 11:37:14.568279 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":152,"threads_started":1}
I20260806 11:37:14.570778 32436 heartbeater.cc:347] Connected to a master server at 127.21.14.254:41705
I20260806 11:37:14.571126 32436 heartbeater.cc:464] Registering TS with master...
I20260806 11:37:14.571681 32436 heartbeater.cc:511] Master 127.21.14.254:41705 requested a full tablet report, sending...
I20260806 11:37:14.573269 31949 ts_manager.cc:194] Registered new tserver with Master: d8d122f3470d4534986c6489c157b0f6 (127.21.14.195:41369)
I20260806 11:37:14.579352 21563 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 11:37:14.583127 31991 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:37:14.581858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48948 (local address 127.21.14.254:41705)
0806 11:37:14.582074 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0806 11:37:14.582078 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:37:14.582333 (+ 255us) server_negotiation.cc:407] Connection header received
0806 11:37:14.582394 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:37:14.582396 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:37:14.582436 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:37:14.582594 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:37:14.582956 (+ 362us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48948: BlockingRecv error: recv got EOF from 127.0.0.1:48948 (error 108)
Metrics: {"server-negotiator.queue_time_us":105}
I20260806 11:37:14.583811 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 32013
I20260806 11:37:14.593524 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 32153
I20260806 11:37:14.601574 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 32297
I20260806 11:37:14.610267 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 31915
2026-08-06T11:37:14Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (948 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 11:37:14 dist-test-slave-7c35 krb5kdc[32447](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 11:37:14 dist-test-slave-7c35 krb5kdc[32447](info): set up 2 sockets
Aug 06 11:37:14 dist-test-slave-7c35 krb5kdc[32447](info): commencing operation
krb5kdc: starting...
W20260806 11:37:15.713863 21563 mini_kdc.cc:121] Time spent starting KDC: real 1.078s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Aug 06 11:37:15 dist-test-slave-7c35 krb5kdc[32447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016235, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T11:37:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T11:37:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.14.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab.
Entry for principal kudu/127.21.14.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab.
I20260806 11:37:15.870983 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:42997
--webserver_interface=127.21.14.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab
--principal=kudu/127.21.14.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:37143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.14.254:42997
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:37:16.002957 32461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:37:16.003369 32461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:37:16.003461 32461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:37:16.008001 32461 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 11:37:16.008126 32461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:37:16.008163 32461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:37:16.008205 32461 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 11:37:16.008245 32461 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 11:37:16.015515 32461 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:37143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.14.254:42997
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab
--principal=kudu/127.21.14.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:42997
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.14.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32461
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:37:16.016831 32461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:37:16.018052 32461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:37:16.028743 32466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:37:16.028780 32467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:37:16.028846 32469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:37:16.029254 32461 server_base.cc:1034] running on GCE node
I20260806 11:37:16.029973 32461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:37:16.031383 32461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:37:16.032617 32461 hybrid_clock.cc:648] HybridClock initialized: now 1786016236032581 us; error 54 us; skew 500 ppm
Aug 06 11:37:16 dist-test-slave-7c35 krb5kdc[32447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016236, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:37:16.035856 32461 init.cc:377] Logged in from keytab as kudu/127.21.14.254@KRBTEST.COM (short username kudu)
I20260806 11:37:16.036934 32461 webserver.cc:460] Webserver started at http://127.21.14.254:34891/ using document root <none> and password file <none>
I20260806 11:37:16.037546 32461 fs_manager.cc:359] Metadata directory not provided
I20260806 11:37:16.037614 32461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:37:16.037804 32461 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:37:16.040203 32461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/minicluster-data/master-0/data/instance:
uuid: "87ec0dcbaddc409b8f4e8be753da01b2"
format_stamp: "Formatted at 2026-08-06 11:37:16 on dist-test-slave-7c35"
server_key: "7f4f5275860b8b790972183d406bac8e"
server_key_iv: "79a3756d7e245abbcbb18a2d0a72066b"
server_key_version: "encryptionkey@0"
I20260806 11:37:16.040788 32461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/minicluster-data/master-0/wal/instance:
uuid: "87ec0dcbaddc409b8f4e8be753da01b2"
format_stamp: "Formatted at 2026-08-06 11:37:16 on dist-test-slave-7c35"
server_key: "7f4f5275860b8b790972183d406bac8e"
server_key_iv: "79a3756d7e245abbcbb18a2d0a72066b"
server_key_version: "encryptionkey@0"
I20260806 11:37:16.045135 32461 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 11:37:16.048309 32476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:37:16.050334 32461 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 11:37:16.050498 32461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/minicluster-data/master-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/minicluster-data/master-0/wal
uuid: "87ec0dcbaddc409b8f4e8be753da01b2"
format_stamp: "Formatted at 2026-08-06 11:37:16 on dist-test-slave-7c35"
server_key: "7f4f5275860b8b790972183d406bac8e"
server_key_iv: "79a3756d7e245abbcbb18a2d0a72066b"
server_key_version: "encryptionkey@0"
I20260806 11:37:16.050625 32461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:37:16.068795 32461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:37:16.072988 32461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:37:16.073190 32461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:37:16.082387 32461 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.254:42997
I20260806 11:37:16.082717 32532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.254:42997 every 8 connection(s)
I20260806 11:37:16.083788 32461 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
I20260806 11:37:16.087457 32534 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:37:16.090502 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 32461
I20260806 11:37:16.090667 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/minicluster-data/master-0/wal/instance
I20260806 11:37:16.091017 21563 external_mini_cluster.cc:1428] Setting key 5565785fac21a153235832176a4186a4
I20260806 11:37:16.095391 32534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87ec0dcbaddc409b8f4e8be753da01b2: Bootstrap starting.
I20260806 11:37:16.098222 32534 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 87ec0dcbaddc409b8f4e8be753da01b2: Neither blocks nor log segments found. Creating new log.
I20260806 11:37:16.099229 32534 log.cc:824] T 00000000000000000000000000000000 P 87ec0dcbaddc409b8f4e8be753da01b2: Log is configured to *not* fsync() on all Append() calls
Aug 06 11:37:16 dist-test-slave-7c35 krb5kdc[32447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016235, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:37:16.102432 32534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87ec0dcbaddc409b8f4e8be753da01b2: No bootstrap required, opened a new log
I20260806 11:37:16.106753 32534 raft_consensus.cc:348] T 00000000000000000000000000000000 P 87ec0dcbaddc409b8f4e8be753da01b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87ec0dcbaddc409b8f4e8be753da01b2" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 42997 } }
I20260806 11:37:16.107154 32534 raft_consensus.cc:374] T 00000000000000000000000000000000 P 87ec0dcbaddc409b8f4e8be753da01b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:37:16.107259 32534 raft_consensus.cc:729] T 00000000000000000000000000000000 P 87ec0dcbaddc409b8f4e8be753da01b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87ec0dcbaddc409b8f4e8be753da01b2, State: Initialized, Role: FOLLOWER
I20260806 11:37:16.107870 32534 consensus_queue.cc:260] T 00000000000000000000000000000000 P 87ec0dcbaddc409b8f4e8be753da01b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87ec0dcbaddc409b8f4e8be753da01b2" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 42997 } }
I20260806 11:37:16.108074 32534 raft_consensus.cc:388] T 00000000000000000000000000000000 P 87ec0dcbaddc409b8f4e8be753da01b2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 11:37:16.108152 32534 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87ec0dcbaddc409b8f4e8be753da01b2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 11:37:16.108251 32534 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 87ec0dcbaddc409b8f4e8be753da01b2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:37:16.109254 32538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:37:16.092777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37630 (local address 127.21.14.254:42997)
0806 11:37:16.093435 (+ 658us) server_negotiation.cc:206] Beginning negotiation
0806 11:37:16.093455 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 11:37:16.093510 (+ 55us) server_negotiation.cc:407] Connection header received
0806 11:37:16.094268 (+ 758us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:37:16.094288 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:37:16.094690 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:37:16.095165 (+ 475us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:37:16.096111 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:16.097338 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:16.098314 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:16.099035 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:16.099233 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:16.102497 (+ 3264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:37:16.102504 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:37:16.102522 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:37:16.102581 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:37:16.105147 (+ 2566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:37:16.105750 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:37:16.105757 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:37:16.105765 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:37:16.105907 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:37:16.106240 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:37:16.106243 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:37:16.106245 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:37:16.106742 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:37:16.107085 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0806 11:37:16.107652 (+ 567us) server_negotiation.cc:299] Negotiation successful
0806 11:37:16.108052 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":475,"thread_start_us":208,"threads_started":1}
I20260806 11:37:16.109690 32534 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87ec0dcbaddc409b8f4e8be753da01b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87ec0dcbaddc409b8f4e8be753da01b2" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 42997 } }
I20260806 11:37:16.110138 32534 leader_election.cc:302] T 00000000000000000000000000000000 P 87ec0dcbaddc409b8f4e8be753da01b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 87ec0dcbaddc409b8f4e8be753da01b2; no voters:
I20260806 11:37:16.110525 32534 leader_election.cc:288] T 00000000000000000000000000000000 P 87ec0dcbaddc409b8f4e8be753da01b2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 11:37:16.110678 32540 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 87ec0dcbaddc409b8f4e8be753da01b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:37:16.110975 32540 raft_consensus.cc:686] T 00000000000000000000000000000000 P 87ec0dcbaddc409b8f4e8be753da01b2 [term 1 LEADER]: Becoming Leader. State: Replica: 87ec0dcbaddc409b8f4e8be753da01b2, State: Running, Role: LEADER
I20260806 11:37:16.111300 32540 consensus_queue.cc:237] T 00000000000000000000000000000000 P 87ec0dcbaddc409b8f4e8be753da01b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87ec0dcbaddc409b8f4e8be753da01b2" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 42997 } }
I20260806 11:37:16.112231 32534 sys_catalog.cc:564] T 00000000000000000000000000000000 P 87ec0dcbaddc409b8f4e8be753da01b2 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 11:37:16.112991 32541 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87ec0dcbaddc409b8f4e8be753da01b2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "87ec0dcbaddc409b8f4e8be753da01b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87ec0dcbaddc409b8f4e8be753da01b2" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 42997 } } }
I20260806 11:37:16.113034 32542 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87ec0dcbaddc409b8f4e8be753da01b2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 87ec0dcbaddc409b8f4e8be753da01b2. Latest consensus state: current_term: 1 leader_uuid: "87ec0dcbaddc409b8f4e8be753da01b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87ec0dcbaddc409b8f4e8be753da01b2" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 42997 } } }
I20260806 11:37:16.113328 32542 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87ec0dcbaddc409b8f4e8be753da01b2 [sys.catalog]: This master's current role is: LEADER
I20260806 11:37:16.113371 32541 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87ec0dcbaddc409b8f4e8be753da01b2 [sys.catalog]: This master's current role is: LEADER
I20260806 11:37:16.113845 32550 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 11:37:16.117349 32550 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 11:37:16.124506 32550 catalog_manager.cc:1347] Generated new cluster ID: 929ce95d1be84f8090a65c27f44225cd
I20260806 11:37:16.124647 32550 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 11:37:16.147195 32550 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 11:37:16.148705 32550 catalog_manager.cc:1504] Loading token signing keys...
I20260806 11:37:16.162415 32550 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 87ec0dcbaddc409b8f4e8be753da01b2: Generated new TSK 0
I20260806 11:37:16.163733 32550 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.14.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab.
Entry for principal kudu/127.21.14.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab.
I20260806 11:37:16.221035 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.193:0
--local_ip_for_outbound_sockets=127.21.14.193
--webserver_interface=127.21.14.193
--webserver_port=0
--tserver_master_addrs=127.21.14.254:42997
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab
--principal=kudu/127.21.14.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:37143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:37:16.361404 32562 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:37:16.361740 32562 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:37:16.361788 32562 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:37:16.367164 32562 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:37:16.367269 32562 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:37:16.367339 32562 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.193
I20260806 11:37:16.373519 32562 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:37143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab
--principal=kudu/127.21.14.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.14.193
--webserver_port=0
--tserver_master_addrs=127.21.14.254:42997
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32562
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.193
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:37:16.374886 32562 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:37:16.376029 32562 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:37:16.387563 32570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:37:16.387564 32572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:37:16.387811 32562 server_base.cc:1034] running on GCE node
W20260806 11:37:16.387599 32569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:37:16.388402 32562 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:37:16.389236 32562 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:37:16.390520 32562 hybrid_clock.cc:648] HybridClock initialized: now 1786016236390498 us; error 42 us; skew 500 ppm
Aug 06 11:37:16 dist-test-slave-7c35 krb5kdc[32447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016236, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:37:16.399875 32562 init.cc:377] Logged in from keytab as kudu/127.21.14.193@KRBTEST.COM (short username kudu)
I20260806 11:37:16.401201 32562 webserver.cc:460] Webserver started at http://127.21.14.193:44999/ using document root <none> and password file <none>
I20260806 11:37:16.401939 32562 fs_manager.cc:359] Metadata directory not provided
I20260806 11:37:16.402060 32562 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:37:16.402341 32562 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:37:16.404937 32562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/minicluster-data/ts-0/data/instance:
uuid: "2ff841dd9de54f9693d49801f47524c5"
format_stamp: "Formatted at 2026-08-06 11:37:16 on dist-test-slave-7c35"
server_key: "08a5b4e7aaecd19ffd811a1fe8aaeeda"
server_key_iv: "1023e402067fd9b29e1014d51791bba4"
server_key_version: "encryptionkey@0"
I20260806 11:37:16.405650 32562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/minicluster-data/ts-0/wal/instance:
uuid: "2ff841dd9de54f9693d49801f47524c5"
format_stamp: "Formatted at 2026-08-06 11:37:16 on dist-test-slave-7c35"
server_key: "08a5b4e7aaecd19ffd811a1fe8aaeeda"
server_key_iv: "1023e402067fd9b29e1014d51791bba4"
server_key_version: "encryptionkey@0"
I20260806 11:37:16.410418 32562 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.006s
I20260806 11:37:16.413722 32579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:37:16.415557 32562 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 11:37:16.415768 32562 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/minicluster-data/ts-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/minicluster-data/ts-0/wal
uuid: "2ff841dd9de54f9693d49801f47524c5"
format_stamp: "Formatted at 2026-08-06 11:37:16 on dist-test-slave-7c35"
server_key: "08a5b4e7aaecd19ffd811a1fe8aaeeda"
server_key_iv: "1023e402067fd9b29e1014d51791bba4"
server_key_version: "encryptionkey@0"
I20260806 11:37:16.415923 32562 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:37:16.433228 32562 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:37:16.437448 32562 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:37:16.437728 32562 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:37:16.438472 32562 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:37:16.439571 32562 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:37:16.439651 32562 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:37:16.439733 32562 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:37:16.439785 32562 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:37:16.451756 32562 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.193:36001
I20260806 11:37:16.451823 32700 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.193:36001 every 8 connection(s)
I20260806 11:37:16.452867 32562 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
I20260806 11:37:16.461788 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 32562
I20260806 11:37:16.462101 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/minicluster-data/ts-0/wal/instance
I20260806 11:37:16.462491 21563 external_mini_cluster.cc:1428] Setting key 228f9ecd80c6fbb5d7ab3035c280c4f0
Aug 06 11:37:16 dist-test-slave-7c35 krb5kdc[32447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016236, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.193@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:37:16.471167 32538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:37:16.455559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:60323 (local address 127.21.14.254:42997)
0806 11:37:16.455752 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 11:37:16.455758 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 11:37:16.456835 (+ 1077us) server_negotiation.cc:407] Connection header received
0806 11:37:16.458545 (+ 1710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:37:16.458548 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:37:16.458622 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:37:16.458708 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:37:16.460357 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:16.460975 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:16.462794 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:16.463161 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:16.463284 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:16.466589 (+ 3305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:37:16.466596 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:37:16.466601 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:37:16.466643 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:37:16.468398 (+ 1755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:37:16.469078 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:37:16.469085 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:37:16.469090 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:37:16.469203 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:37:16.469627 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:37:16.469633 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:37:16.469638 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:37:16.469943 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:37:16.470105 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0806 11:37:16.470743 (+ 638us) server_negotiation.cc:299] Negotiation successful
0806 11:37:16.470948 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 11:37:16.472092 32705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:37:16.456039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:42997 (local address 127.21.14.193:60323)
0806 11:37:16.456653 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0806 11:37:16.456706 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 11:37:16.458215 (+ 1509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:37:16.459085 (+ 870us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:37:16.459094 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:37:16.459538 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:37:16.460135 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:16.460153 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:16.461190 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:16.461193 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:16.462607 (+ 1414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:16.462625 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:16.463447 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:16.463453 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:16.463696 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:16.464504 (+ 808us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:37:16.464523 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:37:16.466308 (+ 1785us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:37:16.468573 (+ 2265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:37:16.468579 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:37:16.468597 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:37:16.468938 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:37:16.469318 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:37:16.469320 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:37:16.469322 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:37:16.469485 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:37:16.470120 (+ 635us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:37:16.470126 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:37:16.470447 (+ 321us) client_negotiation.cc:769] Sending connection context
0806 11:37:16.470642 (+ 195us) client_negotiation.cc:240] Negotiation successful
0806 11:37:16.471081 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":155,"threads_started":1}
I20260806 11:37:16.473810 32701 heartbeater.cc:347] Connected to a master server at 127.21.14.254:42997
I20260806 11:37:16.474112 32701 heartbeater.cc:464] Registering TS with master...
I20260806 11:37:16.474709 32701 heartbeater.cc:511] Master 127.21.14.254:42997 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.21.14.194@KRBTEST.COM; defaulting to no policy
I20260806 11:37:16.476653 32495 ts_manager.cc:194] Registered new tserver with Master: 2ff841dd9de54f9693d49801f47524c5 (127.21.14.193:36001)
I20260806 11:37:16.478056 32495 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.14.193@KRBTEST.COM'} at 127.21.14.193:60323
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab.
Entry for principal kudu/127.21.14.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab.
I20260806 11:37:16.504858 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.194:0
--local_ip_for_outbound_sockets=127.21.14.194
--webserver_interface=127.21.14.194
--webserver_port=0
--tserver_master_addrs=127.21.14.254:42997
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab
--principal=kudu/127.21.14.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:37143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:37:16.638705 32708 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:37:16.639119 32708 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:37:16.639174 32708 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:37:16.643507 32708 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:37:16.643580 32708 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:37:16.643643 32708 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.194
I20260806 11:37:16.649326 32708 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:37143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab
--principal=kudu/127.21.14.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.14.194
--webserver_port=0
--tserver_master_addrs=127.21.14.254:42997
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32708
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.194
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:37:16.650538 32708 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:37:16.651659 32708 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:37:16.663789 32716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:37:16.663789 32714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:37:16.663874 32713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:37:16.664059 32708 server_base.cc:1034] running on GCE node
I20260806 11:37:16.664702 32708 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:37:16.665690 32708 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:37:16.666989 32708 hybrid_clock.cc:648] HybridClock initialized: now 1786016236666970 us; error 39 us; skew 500 ppm
Aug 06 11:37:16 dist-test-slave-7c35 krb5kdc[32447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016236, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:37:16.670468 32708 init.cc:377] Logged in from keytab as kudu/127.21.14.194@KRBTEST.COM (short username kudu)
I20260806 11:37:16.671645 32708 webserver.cc:460] Webserver started at http://127.21.14.194:40295/ using document root <none> and password file <none>
I20260806 11:37:16.672310 32708 fs_manager.cc:359] Metadata directory not provided
I20260806 11:37:16.672428 32708 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:37:16.672700 32708 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:37:16.675287 32708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/minicluster-data/ts-1/data/instance:
uuid: "9aa4eeda4bc94ea296275e7e6ea65886"
format_stamp: "Formatted at 2026-08-06 11:37:16 on dist-test-slave-7c35"
server_key: "4ae1de2aef20d4235fe642cd79dbbce2"
server_key_iv: "23389bdc1c56b3da2f57757e92163899"
server_key_version: "encryptionkey@0"
I20260806 11:37:16.676021 32708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/minicluster-data/ts-1/wal/instance:
uuid: "9aa4eeda4bc94ea296275e7e6ea65886"
format_stamp: "Formatted at 2026-08-06 11:37:16 on dist-test-slave-7c35"
server_key: "4ae1de2aef20d4235fe642cd79dbbce2"
server_key_iv: "23389bdc1c56b3da2f57757e92163899"
server_key_version: "encryptionkey@0"
I20260806 11:37:16.681195 32708 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 11:37:16.684273 32724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:37:16.685667 32708 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 11:37:16.685835 32708 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/minicluster-data/ts-1/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/minicluster-data/ts-1/wal
uuid: "9aa4eeda4bc94ea296275e7e6ea65886"
format_stamp: "Formatted at 2026-08-06 11:37:16 on dist-test-slave-7c35"
server_key: "4ae1de2aef20d4235fe642cd79dbbce2"
server_key_iv: "23389bdc1c56b3da2f57757e92163899"
server_key_version: "encryptionkey@0"
I20260806 11:37:16.685979 32708 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:37:16.698698 32708 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:37:16.702618 32708 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:37:16.702953 32708 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:37:16.703655 32708 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:37:16.704692 32708 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:37:16.704770 32708 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:37:16.704844 32708 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:37:16.704896 32708 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:37:16.716559 32708 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.194:40799
I20260806 11:37:16.716622 376 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.194:40799 every 8 connection(s)
I20260806 11:37:16.717624 32708 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
I20260806 11:37:16.724687 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 32708
I20260806 11:37:16.724984 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/minicluster-data/ts-1/wal/instance
I20260806 11:37:16.725518 21563 external_mini_cluster.cc:1428] Setting key 60cbf400c50afe0975cc68e753f196c8
Aug 06 11:37:16 dist-test-slave-7c35 krb5kdc[32447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016236, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.194@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:37:16.736343 32538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:37:16.721678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:60803 (local address 127.21.14.254:42997)
0806 11:37:16.721879 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0806 11:37:16.721883 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:37:16.721900 (+ 17us) server_negotiation.cc:407] Connection header received
0806 11:37:16.723283 (+ 1383us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:37:16.723286 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:37:16.723342 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:37:16.723433 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:37:16.725721 (+ 2288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:16.726437 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:16.728028 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:16.728441 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:16.728574 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:16.731965 (+ 3391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:37:16.731969 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:37:16.731971 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:37:16.732002 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:37:16.733623 (+ 1621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:37:16.734300 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:37:16.734303 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:37:16.734305 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:37:16.734390 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:37:16.734808 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:37:16.734811 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:37:16.734813 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:37:16.735092 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:37:16.735216 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 11:37:16.735896 (+ 680us) server_negotiation.cc:299] Negotiation successful
0806 11:37:16.736161 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
WARNING: no policy specified for kudu/127.21.14.195@KRBTEST.COM; defaulting to no policy
I20260806 11:37:16.737588 380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:37:16.720521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:42997 (local address 127.21.14.194:60803)
0806 11:37:16.721192 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0806 11:37:16.721245 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 11:37:16.722976 (+ 1731us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:37:16.723816 (+ 840us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:37:16.723826 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:37:16.724285 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:37:16.725438 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:16.725480 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:16.726606 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:16.726610 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:16.727851 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:16.727872 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:16.728627 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:16.728630 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:16.728801 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:16.729579 (+ 778us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:37:16.729626 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:37:16.731729 (+ 2103us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:37:16.733783 (+ 2054us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:37:16.733791 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:37:16.733814 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:37:16.734160 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:37:16.734498 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:37:16.734501 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:37:16.734503 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:37:16.734692 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:37:16.735220 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:37:16.735232 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:37:16.735605 (+ 373us) client_negotiation.cc:769] Sending connection context
0806 11:37:16.735831 (+ 226us) client_negotiation.cc:240] Negotiation successful
0806 11:37:16.736366 (+ 535us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":148,"threads_started":1}
I20260806 11:37:16.739368 377 heartbeater.cc:347] Connected to a master server at 127.21.14.254:42997
I20260806 11:37:16.739645 377 heartbeater.cc:464] Registering TS with master...
I20260806 11:37:16.740343 377 heartbeater.cc:511] Master 127.21.14.254:42997 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.195@KRBTEST.COM" created.
I20260806 11:37:16.742036 32495 ts_manager.cc:194] Registered new tserver with Master: 9aa4eeda4bc94ea296275e7e6ea65886 (127.21.14.194:40799)
I20260806 11:37:16.742705 32495 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.14.194@KRBTEST.COM'} at 127.21.14.194:60803
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab.
Entry for principal kudu/127.21.14.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab.
I20260806 11:37:16.764490 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.195:0
--local_ip_for_outbound_sockets=127.21.14.195
--webserver_interface=127.21.14.195
--webserver_port=0
--tserver_master_addrs=127.21.14.254:42997
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab
--principal=kudu/127.21.14.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:37143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:37:16.894374 383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:37:16.894721 383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:37:16.894773 383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:37:16.899895 383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:37:16.900005 383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:37:16.900072 383 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.195
I20260806 11:37:16.906014 383 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:37143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab
--principal=kudu/127.21.14.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.14.195
--webserver_port=0
--tserver_master_addrs=127.21.14.254:42997
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.383
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.195
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:37:16.907370 383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:37:16.908541 383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 11:37:16.919878 383 server_base.cc:1034] running on GCE node
W20260806 11:37:16.919833 388 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:37:16.919833 389 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:37:16.919859 391 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:37:16.920706 383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:37:16.921499 383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:37:16.922703 383 hybrid_clock.cc:648] HybridClock initialized: now 1786016236922680 us; error 59 us; skew 500 ppm
Aug 06 11:37:16 dist-test-slave-7c35 krb5kdc[32447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016236, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:37:16.926314 383 init.cc:377] Logged in from keytab as kudu/127.21.14.195@KRBTEST.COM (short username kudu)
I20260806 11:37:16.927496 383 webserver.cc:460] Webserver started at http://127.21.14.195:38517/ using document root <none> and password file <none>
I20260806 11:37:16.928189 383 fs_manager.cc:359] Metadata directory not provided
I20260806 11:37:16.928298 383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:37:16.928565 383 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:37:16.931142 383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/minicluster-data/ts-2/data/instance:
uuid: "25cbff182913473d8f139a262c007dcb"
format_stamp: "Formatted at 2026-08-06 11:37:16 on dist-test-slave-7c35"
server_key: "8f5799a0f7770451e1e17ffcbaa20e35"
server_key_iv: "423aed0285cea726c5cedc0d24ffdee9"
server_key_version: "encryptionkey@0"
I20260806 11:37:16.931913 383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/minicluster-data/ts-2/wal/instance:
uuid: "25cbff182913473d8f139a262c007dcb"
format_stamp: "Formatted at 2026-08-06 11:37:16 on dist-test-slave-7c35"
server_key: "8f5799a0f7770451e1e17ffcbaa20e35"
server_key_iv: "423aed0285cea726c5cedc0d24ffdee9"
server_key_version: "encryptionkey@0"
I20260806 11:37:16.936719 383 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 11:37:16.939833 398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:37:16.941349 383 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 11:37:16.941536 383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/minicluster-data/ts-2/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/minicluster-data/ts-2/wal
uuid: "25cbff182913473d8f139a262c007dcb"
format_stamp: "Formatted at 2026-08-06 11:37:16 on dist-test-slave-7c35"
server_key: "8f5799a0f7770451e1e17ffcbaa20e35"
server_key_iv: "423aed0285cea726c5cedc0d24ffdee9"
server_key_version: "encryptionkey@0"
I20260806 11:37:16.941679 383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:37:16.964399 383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:37:16.968010 383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:37:16.968276 383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:37:16.969058 383 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:37:16.970038 383 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:37:16.970139 383 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:37:16.970218 383 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:37:16.970263 383 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:37:16.980616 383 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.195:38079
I20260806 11:37:16.980636 521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.195:38079 every 8 connection(s)
I20260806 11:37:16.981694 383 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
I20260806 11:37:16.983639 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 383
I20260806 11:37:16.983771 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/minicluster-data/ts-2/wal/instance
I20260806 11:37:16.984117 21563 external_mini_cluster.cc:1428] Setting key a57db38add5d2e7bcbcb55d69088241f
Aug 06 11:37:16 dist-test-slave-7c35 krb5kdc[32447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016236, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.195@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:37:16.997417 32538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:37:16.983840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:55331 (local address 127.21.14.254:42997)
0806 11:37:16.984035 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0806 11:37:16.984042 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 11:37:16.985097 (+ 1055us) server_negotiation.cc:407] Connection header received
0806 11:37:16.986689 (+ 1592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:37:16.986692 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:37:16.986743 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:37:16.986821 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:37:16.988234 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:16.988781 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:16.990168 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:16.990591 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:16.990711 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:16.993661 (+ 2950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:37:16.993665 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:37:16.993668 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:37:16.993706 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:37:16.995049 (+ 1343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:37:16.995618 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:37:16.995622 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:37:16.995624 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:37:16.995714 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:37:16.996097 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:37:16.996101 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:37:16.996103 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:37:16.996325 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:37:16.996437 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 11:37:16.997094 (+ 657us) server_negotiation.cc:299] Negotiation successful
0806 11:37:16.997254 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 11:37:16.998549 524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:37:16.984350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:42997 (local address 127.21.14.195:55331)
0806 11:37:16.984925 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0806 11:37:16.984986 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0806 11:37:16.986436 (+ 1450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:37:16.987082 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:37:16.987092 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:37:16.987493 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:37:16.988064 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:16.988080 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:16.988906 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:16.988909 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:16.989990 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:16.990001 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:16.990882 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:16.990889 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:16.991166 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:16.991845 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:37:16.991864 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:37:16.993434 (+ 1570us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:37:16.995174 (+ 1740us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:37:16.995182 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:37:16.995201 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:37:16.995490 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:37:16.995814 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:37:16.995817 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:37:16.995819 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:37:16.995982 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:37:16.996437 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:37:16.996445 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:37:16.996782 (+ 337us) client_negotiation.cc:769] Sending connection context
0806 11:37:16.997023 (+ 241us) client_negotiation.cc:240] Negotiation successful
0806 11:37:16.997406 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":175,"threads_started":1}
I20260806 11:37:16.999819 522 heartbeater.cc:347] Connected to a master server at 127.21.14.254:42997
I20260806 11:37:17.000063 522 heartbeater.cc:464] Registering TS with master...
I20260806 11:37:17.000572 522 heartbeater.cc:511] Master 127.21.14.254:42997 requested a full tablet report, sending...
I20260806 11:37:17.001646 32495 ts_manager.cc:194] Registered new tserver with Master: 25cbff182913473d8f139a262c007dcb (127.21.14.195:38079)
I20260806 11:37:17.002287 32495 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.14.195@KRBTEST.COM'} at 127.21.14.195:55331
I20260806 11:37:17.010756 21563 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 11:37:17.021772 32538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:37:17.013067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37640 (local address 127.21.14.254:42997)
0806 11:37:17.013233 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 11:37:17.013238 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:37:17.013424 (+ 186us) server_negotiation.cc:407] Connection header received
0806 11:37:17.013567 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:37:17.013569 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:37:17.013617 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:37:17.013725 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:37:17.014519 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:17.015212 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:17.016182 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:17.016418 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:17.016508 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:17.017978 (+ 1470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:37:17.017981 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:37:17.017983 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:37:17.018010 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:37:17.019725 (+ 1715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:37:17.020182 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:37:17.020185 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:37:17.020186 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:37:17.020272 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:37:17.020578 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:37:17.020581 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:37:17.020583 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:37:17.020950 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:37:17.021090 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 11:37:17.021386 (+ 296us) server_negotiation.cc:299] Negotiation successful
0806 11:37:17.021513 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 11:37:17.025406 32495 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37640:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 11:37:17.028823 32495 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 11:37:17.045657 537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:37:17.040293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40496 (local address 127.21.14.194:40799)
0806 11:37:17.040910 (+ 617us) server_negotiation.cc:206] Beginning negotiation
0806 11:37:17.040922 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 11:37:17.040954 (+ 32us) server_negotiation.cc:407] Connection header received
0806 11:37:17.041035 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:37:17.041040 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:37:17.041252 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:37:17.041418 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:37:17.041627 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:17.042532 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:17.044003 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:17.044828 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:17.044962 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:17.045197 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0806 11:37:17.045360 (+ 163us) server_negotiation.cc:299] Negotiation successful
0806 11:37:17.045472 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":511,"thread_start_us":118,"threads_started":1}
I20260806 11:37:17.045656 535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:37:17.040085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.194:40799 (local address 127.0.0.1:40496)
0806 11:37:17.040423 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0806 11:37:17.040437 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 11:37:17.040516 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:37:17.041391 (+ 875us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:37:17.041395 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:37:17.041413 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:37:17.041529 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:17.041536 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:17.042686 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:17.042690 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:17.043837 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:17.043849 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:17.044964 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:17.044968 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:17.045119 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:17.045167 (+ 48us) client_negotiation.cc:769] Sending connection context
0806 11:37:17.045327 (+ 160us) client_negotiation.cc:240] Negotiation successful
0806 11:37:17.045480 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"mutex_wait_us":3,"thread_start_us":116,"threads_started":1}
I20260806 11:37:17.045881 536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:37:17.040117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.193:36001 (local address 127.0.0.1:56148)
0806 11:37:17.040819 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0806 11:37:17.040833 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 11:37:17.040935 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:37:17.041710 (+ 775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:37:17.041714 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:37:17.041731 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:37:17.041815 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:17.041820 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:17.043047 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:17.043051 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:17.044169 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:17.044178 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:17.045423 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:17.045426 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:17.045528 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:17.045549 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 11:37:17.045648 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 11:37:17.045735 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":639,"mutex_wait_us":4,"thread_start_us":106,"threads_started":1}
I20260806 11:37:17.045964 538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:37:17.040811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56148 (local address 127.21.14.193:36001)
0806 11:37:17.041324 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0806 11:37:17.041330 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 11:37:17.041353 (+ 23us) server_negotiation.cc:407] Connection header received
0806 11:37:17.041426 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:37:17.041431 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:37:17.041600 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:37:17.041734 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:37:17.041918 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:17.042924 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:17.044316 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:17.045116 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:17.045215 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:17.045350 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 11:37:17.045702 (+ 352us) server_negotiation.cc:299] Negotiation successful
0806 11:37:17.045805 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":79,"threads_started":1}
I20260806 11:37:17.046419 533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:37:17.039267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.195:38079 (local address 127.0.0.1:58416)
0806 11:37:17.039687 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0806 11:37:17.039740 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 11:37:17.039893 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:37:17.040729 (+ 836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:37:17.040736 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:37:17.040762 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:37:17.040910 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:17.040920 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:17.042401 (+ 1481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:17.042405 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:17.043930 (+ 1525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:17.043943 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:17.045301 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:17.045305 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:17.045413 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:17.046104 (+ 691us) client_negotiation.cc:769] Sending connection context
0806 11:37:17.046188 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 11:37:17.046273 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"mutex_wait_us":633,"thread_start_us":138,"threads_started":1}
I20260806 11:37:17.046576 534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:37:17.039380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58416 (local address 127.21.14.195:38079)
0806 11:37:17.039729 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0806 11:37:17.039742 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 11:37:17.039843 (+ 101us) server_negotiation.cc:407] Connection header received
0806 11:37:17.040060 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:37:17.040068 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:37:17.040300 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:37:17.040477 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:37:17.041065 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:17.042250 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:17.044087 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:17.044972 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:17.045091 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:17.045497 (+ 406us) server_negotiation.cc:1035] Waiting for connection context
0806 11:37:17.046249 (+ 752us) server_negotiation.cc:299] Negotiation successful
0806 11:37:17.046407 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":142,"threads_started":1}
I20260806 11:37:17.049434 32630 tablet_service.cc:1467] Processing CreateTablet for tablet 4722a1e5603a4fdf8081eb3ffaf52b92 (DEFAULT_TABLE table=test-table [id=8b40630decde488f870d63bf5e5c8137]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:37:17.049459 306 tablet_service.cc:1467] Processing CreateTablet for tablet 4722a1e5603a4fdf8081eb3ffaf52b92 (DEFAULT_TABLE table=test-table [id=8b40630decde488f870d63bf5e5c8137]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:37:17.050558 32630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4722a1e5603a4fdf8081eb3ffaf52b92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:37:17.050590 306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4722a1e5603a4fdf8081eb3ffaf52b92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:37:17.050410 451 tablet_service.cc:1467] Processing CreateTablet for tablet 4722a1e5603a4fdf8081eb3ffaf52b92 (DEFAULT_TABLE table=test-table [id=8b40630decde488f870d63bf5e5c8137]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:37:17.051461 451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4722a1e5603a4fdf8081eb3ffaf52b92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:37:17.057936 540 tablet_bootstrap.cc:492] T 4722a1e5603a4fdf8081eb3ffaf52b92 P 2ff841dd9de54f9693d49801f47524c5: Bootstrap starting.
I20260806 11:37:17.060048 541 tablet_bootstrap.cc:492] T 4722a1e5603a4fdf8081eb3ffaf52b92 P 9aa4eeda4bc94ea296275e7e6ea65886: Bootstrap starting.
I20260806 11:37:17.060535 542 tablet_bootstrap.cc:492] T 4722a1e5603a4fdf8081eb3ffaf52b92 P 25cbff182913473d8f139a262c007dcb: Bootstrap starting.
I20260806 11:37:17.061149 540 tablet_bootstrap.cc:654] T 4722a1e5603a4fdf8081eb3ffaf52b92 P 2ff841dd9de54f9693d49801f47524c5: Neither blocks nor log segments found. Creating new log.
I20260806 11:37:17.062182 540 log.cc:824] T 4722a1e5603a4fdf8081eb3ffaf52b92 P 2ff841dd9de54f9693d49801f47524c5: Log is configured to *not* fsync() on all Append() calls
I20260806 11:37:17.063194 541 tablet_bootstrap.cc:654] T 4722a1e5603a4fdf8081eb3ffaf52b92 P 9aa4eeda4bc94ea296275e7e6ea65886: Neither blocks nor log segments found. Creating new log.
I20260806 11:37:17.063241 542 tablet_bootstrap.cc:654] T 4722a1e5603a4fdf8081eb3ffaf52b92 P 25cbff182913473d8f139a262c007dcb: Neither blocks nor log segments found. Creating new log.
I20260806 11:37:17.064291 542 log.cc:824] T 4722a1e5603a4fdf8081eb3ffaf52b92 P 25cbff182913473d8f139a262c007dcb: Log is configured to *not* fsync() on all Append() calls
I20260806 11:37:17.064584 541 log.cc:824] T 4722a1e5603a4fdf8081eb3ffaf52b92 P 9aa4eeda4bc94ea296275e7e6ea65886: Log is configured to *not* fsync() on all Append() calls
I20260806 11:37:17.064880 540 tablet_bootstrap.cc:492] T 4722a1e5603a4fdf8081eb3ffaf52b92 P 2ff841dd9de54f9693d49801f47524c5: No bootstrap required, opened a new log
I20260806 11:37:17.065121 540 ts_tablet_manager.cc:1397] T 4722a1e5603a4fdf8081eb3ffaf52b92 P 2ff841dd9de54f9693d49801f47524c5: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260806 11:37:17.067384 541 tablet_bootstrap.cc:492] T 4722a1e5603a4fdf8081eb3ffaf52b92 P 9aa4eeda4bc94ea296275e7e6ea65886: No bootstrap required, opened a new log
I20260806 11:37:17.067682 541 ts_tablet_manager.cc:1397] T 4722a1e5603a4fdf8081eb3ffaf52b92 P 9aa4eeda4bc94ea296275e7e6ea65886: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260806 11:37:17.068949 542 tablet_bootstrap.cc:492] T 4722a1e5603a4fdf8081eb3ffaf52b92 P 25cbff182913473d8f139a262c007dcb: No bootstrap required, opened a new log
I20260806 11:37:17.069224 542 ts_tablet_manager.cc:1397] T 4722a1e5603a4fdf8081eb3ffaf52b92 P 25cbff182913473d8f139a262c007dcb: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260806 11:37:17.070346 540 raft_consensus.cc:348] T 4722a1e5603a4fdf8081eb3ffaf52b92 P 2ff841dd9de54f9693d49801f47524c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25cbff182913473d8f139a262c007dcb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38079 } } peers { permanent_uuid: "2ff841dd9de54f9693d49801f47524c5" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 36001 } } peers { permanent_uuid: "9aa4eeda4bc94ea296275e7e6ea65886" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 40799 } }
I20260806 11:37:17.070888 540 raft_consensus.cc:374] T 4722a1e5603a4fdf8081eb3ffaf52b92 P 2ff841dd9de54f9693d49801f47524c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:37:17.071017 540 raft_consensus.cc:729] T 4722a1e5603a4fdf8081eb3ffaf52b92 P 2ff841dd9de54f9693d49801f47524c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ff841dd9de54f9693d49801f47524c5, State: Initialized, Role: FOLLOWER
I20260806 11:37:17.071702 540 consensus_queue.cc:260] T 4722a1e5603a4fdf8081eb3ffaf52b92 P 2ff841dd9de54f9693d49801f47524c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25cbff182913473d8f139a262c007dcb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38079 } } peers { permanent_uuid: "2ff841dd9de54f9693d49801f47524c5" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 36001 } } peers { permanent_uuid: "9aa4eeda4bc94ea296275e7e6ea65886" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 40799 } }
I20260806 11:37:17.072028 541 raft_consensus.cc:348] T 4722a1e5603a4fdf8081eb3ffaf52b92 P 9aa4eeda4bc94ea296275e7e6ea65886 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25cbff182913473d8f139a262c007dcb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38079 } } peers { permanent_uuid: "2ff841dd9de54f9693d49801f47524c5" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 36001 } } peers { permanent_uuid: "9aa4eeda4bc94ea296275e7e6ea65886" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 40799 } }
I20260806 11:37:17.072619 32701 heartbeater.cc:503] Master 127.21.14.254:42997 was elected leader, sending a full tablet report...
I20260806 11:37:17.072999 540 ts_tablet_manager.cc:1428] T 4722a1e5603a4fdf8081eb3ffaf52b92 P 2ff841dd9de54f9693d49801f47524c5: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260806 11:37:17.073132 541 raft_consensus.cc:374] T 4722a1e5603a4fdf8081eb3ffaf52b92 P 9aa4eeda4bc94ea296275e7e6ea65886 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:37:17.073295 541 raft_consensus.cc:729] T 4722a1e5603a4fdf8081eb3ffaf52b92 P 9aa4eeda4bc94ea296275e7e6ea65886 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aa4eeda4bc94ea296275e7e6ea65886, State: Initialized, Role: FOLLOWER
I20260806 11:37:17.073876 541 consensus_queue.cc:260] T 4722a1e5603a4fdf8081eb3ffaf52b92 P 9aa4eeda4bc94ea296275e7e6ea65886 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25cbff182913473d8f139a262c007dcb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38079 } } peers { permanent_uuid: "2ff841dd9de54f9693d49801f47524c5" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 36001 } } peers { permanent_uuid: "9aa4eeda4bc94ea296275e7e6ea65886" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 40799 } }
I20260806 11:37:17.074275 542 raft_consensus.cc:348] T 4722a1e5603a4fdf8081eb3ffaf52b92 P 25cbff182913473d8f139a262c007dcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25cbff182913473d8f139a262c007dcb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38079 } } peers { permanent_uuid: "2ff841dd9de54f9693d49801f47524c5" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 36001 } } peers { permanent_uuid: "9aa4eeda4bc94ea296275e7e6ea65886" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 40799 } }
I20260806 11:37:17.074693 377 heartbeater.cc:503] Master 127.21.14.254:42997 was elected leader, sending a full tablet report...
I20260806 11:37:17.074766 542 raft_consensus.cc:374] T 4722a1e5603a4fdf8081eb3ffaf52b92 P 25cbff182913473d8f139a262c007dcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:37:17.074882 542 raft_consensus.cc:729] T 4722a1e5603a4fdf8081eb3ffaf52b92 P 25cbff182913473d8f139a262c007dcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25cbff182913473d8f139a262c007dcb, State: Initialized, Role: FOLLOWER
I20260806 11:37:17.075405 542 consensus_queue.cc:260] T 4722a1e5603a4fdf8081eb3ffaf52b92 P 25cbff182913473d8f139a262c007dcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25cbff182913473d8f139a262c007dcb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38079 } } peers { permanent_uuid: "2ff841dd9de54f9693d49801f47524c5" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 36001 } } peers { permanent_uuid: "9aa4eeda4bc94ea296275e7e6ea65886" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 40799 } }
I20260806 11:37:17.076542 522 heartbeater.cc:503] Master 127.21.14.254:42997 was elected leader, sending a full tablet report...
I20260806 11:37:17.076651 542 ts_tablet_manager.cc:1428] T 4722a1e5603a4fdf8081eb3ffaf52b92 P 25cbff182913473d8f139a262c007dcb: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260806 11:37:17.077392 541 ts_tablet_manager.cc:1428] T 4722a1e5603a4fdf8081eb3ffaf52b92 P 9aa4eeda4bc94ea296275e7e6ea65886: Time spent starting tablet: real 0.010s user 0.007s sys 0.000s
W20260806 11:37:17.204668 32703 tablet.cc:2369] T 4722a1e5603a4fdf8081eb3ffaf52b92 P 2ff841dd9de54f9693d49801f47524c5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 11:37:17.219074 378 tablet.cc:2369] T 4722a1e5603a4fdf8081eb3ffaf52b92 P 9aa4eeda4bc94ea296275e7e6ea65886: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 11:37:17.228346 548 raft_consensus.cc:482] T 4722a1e5603a4fdf8081eb3ffaf52b92 P 25cbff182913473d8f139a262c007dcb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 11:37:17.228606 548 raft_consensus.cc:504] T 4722a1e5603a4fdf8081eb3ffaf52b92 P 25cbff182913473d8f139a262c007dcb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25cbff182913473d8f139a262c007dcb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38079 } } peers { permanent_uuid: "2ff841dd9de54f9693d49801f47524c5" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 36001 } } peers { permanent_uuid: "9aa4eeda4bc94ea296275e7e6ea65886" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 40799 } }
I20260806 11:37:17.230020 548 leader_election.cc:288] T 4722a1e5603a4fdf8081eb3ffaf52b92 P 25cbff182913473d8f139a262c007dcb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ff841dd9de54f9693d49801f47524c5 (127.21.14.193:36001), 9aa4eeda4bc94ea296275e7e6ea65886 (127.21.14.194:40799)
W20260806 11:37:17.232728 523 tablet.cc:2369] T 4722a1e5603a4fdf8081eb3ffaf52b92 P 25cbff182913473d8f139a262c007dcb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 11:37:17.234360 538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:37:17.230435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:55613 (local address 127.21.14.193:36001)
0806 11:37:17.230592 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 11:37:17.230599 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 11:37:17.230617 (+ 18us) server_negotiation.cc:407] Connection header received
0806 11:37:17.230815 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:37:17.230818 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:37:17.230935 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:37:17.231059 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:37:17.231358 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:17.231888 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:17.233234 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:17.233725 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:17.233874 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:17.233935 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 11:37:17.234138 (+ 203us) server_negotiation.cc:299] Negotiation successful
0806 11:37:17.234220 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 11:37:17.234375 524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:37:17.230289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.193:36001 (local address 127.21.14.195:55613)
0806 11:37:17.230548 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0806 11:37:17.230565 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 11:37:17.230681 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:37:17.231076 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:37:17.231080 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:37:17.231100 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:37:17.231231 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:17.231237 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:17.232023 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:17.232027 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:17.233066 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:17.233079 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:17.233874 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:17.233877 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:17.233992 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:17.234015 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 11:37:17.234117 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 11:37:17.234207 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169}
I20260806 11:37:17.234436 537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:37:17.230547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:47615 (local address 127.21.14.194:40799)
0806 11:37:17.230690 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 11:37:17.230695 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:37:17.230989 (+ 294us) server_negotiation.cc:407] Connection header received
0806 11:37:17.231147 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:37:17.231150 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:37:17.231197 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:37:17.231291 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:37:17.231585 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:17.232093 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:17.233331 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:17.233797 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:17.233945 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:17.234008 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 11:37:17.234215 (+ 207us) server_negotiation.cc:299] Negotiation successful
0806 11:37:17.234299 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 11:37:17.234509 549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:37:17.230405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.194:40799 (local address 127.21.14.195:47615)
0806 11:37:17.230837 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0806 11:37:17.230870 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 11:37:17.231024 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:37:17.231301 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:37:17.231304 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:37:17.231319 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:37:17.231456 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:17.231462 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:17.232223 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:17.232230 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:17.233180 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:37:17.233194 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:37:17.233938 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:37:17.233942 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:37:17.234082 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:17.234101 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 11:37:17.234195 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 11:37:17.234275 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":358,"mutex_wait_us":1,"thread_start_us":151,"threads_started":1}
I20260806 11:37:17.235287 32650 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4722a1e5603a4fdf8081eb3ffaf52b92" candidate_uuid: "25cbff182913473d8f139a262c007dcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ff841dd9de54f9693d49801f47524c5" is_pre_election: true
I20260806 11:37:17.235286 328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4722a1e5603a4fdf8081eb3ffaf52b92" candidate_uuid: "25cbff182913473d8f139a262c007dcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa4eeda4bc94ea296275e7e6ea65886" is_pre_election: true
I20260806 11:37:17.235694 328 raft_consensus.cc:2454] T 4722a1e5603a4fdf8081eb3ffaf52b92 P 9aa4eeda4bc94ea296275e7e6ea65886 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25cbff182913473d8f139a262c007dcb in term 0.
I20260806 11:37:17.235693 32650 raft_consensus.cc:2454] T 4722a1e5603a4fdf8081eb3ffaf52b92 P 2ff841dd9de54f9693d49801f47524c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25cbff182913473d8f139a262c007dcb in term 0.
I20260806 11:37:17.236334 399 leader_election.cc:302] T 4722a1e5603a4fdf8081eb3ffaf52b92 P 25cbff182913473d8f139a262c007dcb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 25cbff182913473d8f139a262c007dcb, 2ff841dd9de54f9693d49801f47524c5, 9aa4eeda4bc94ea296275e7e6ea65886; no voters:
I20260806 11:37:17.236680 548 raft_consensus.cc:2781] T 4722a1e5603a4fdf8081eb3ffaf52b92 P 25cbff182913473d8f139a262c007dcb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 11:37:17.236785 548 raft_consensus.cc:482] T 4722a1e5603a4fdf8081eb3ffaf52b92 P 25cbff182913473d8f139a262c007dcb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 11:37:17.236833 548 raft_consensus.cc:3037] T 4722a1e5603a4fdf8081eb3ffaf52b92 P 25cbff182913473d8f139a262c007dcb [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:37:17.237986 548 raft_consensus.cc:504] T 4722a1e5603a4fdf8081eb3ffaf52b92 P 25cbff182913473d8f139a262c007dcb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25cbff182913473d8f139a262c007dcb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38079 } } peers { permanent_uuid: "2ff841dd9de54f9693d49801f47524c5" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 36001 } } peers { permanent_uuid: "9aa4eeda4bc94ea296275e7e6ea65886" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 40799 } }
I20260806 11:37:17.239075 548 leader_election.cc:288] T 4722a1e5603a4fdf8081eb3ffaf52b92 P 25cbff182913473d8f139a262c007dcb [CANDIDATE]: Term 1 election: Requested vote from peers 2ff841dd9de54f9693d49801f47524c5 (127.21.14.193:36001), 9aa4eeda4bc94ea296275e7e6ea65886 (127.21.14.194:40799)
I20260806 11:37:17.239506 32650 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4722a1e5603a4fdf8081eb3ffaf52b92" candidate_uuid: "25cbff182913473d8f139a262c007dcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ff841dd9de54f9693d49801f47524c5"
I20260806 11:37:17.239650 328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4722a1e5603a4fdf8081eb3ffaf52b92" candidate_uuid: "25cbff182913473d8f139a262c007dcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa4eeda4bc94ea296275e7e6ea65886"
I20260806 11:37:17.239725 32650 raft_consensus.cc:3037] T 4722a1e5603a4fdf8081eb3ffaf52b92 P 2ff841dd9de54f9693d49801f47524c5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:37:17.239850 328 raft_consensus.cc:3037] T 4722a1e5603a4fdf8081eb3ffaf52b92 P 9aa4eeda4bc94ea296275e7e6ea65886 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:37:17.241410 32650 raft_consensus.cc:2454] T 4722a1e5603a4fdf8081eb3ffaf52b92 P 2ff841dd9de54f9693d49801f47524c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25cbff182913473d8f139a262c007dcb in term 1.
I20260806 11:37:17.241412 328 raft_consensus.cc:2454] T 4722a1e5603a4fdf8081eb3ffaf52b92 P 9aa4eeda4bc94ea296275e7e6ea65886 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25cbff182913473d8f139a262c007dcb in term 1.
I20260806 11:37:17.241807 399 leader_election.cc:302] T 4722a1e5603a4fdf8081eb3ffaf52b92 P 25cbff182913473d8f139a262c007dcb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 25cbff182913473d8f139a262c007dcb, 2ff841dd9de54f9693d49801f47524c5, 9aa4eeda4bc94ea296275e7e6ea65886; no voters:
I20260806 11:37:17.242055 548 raft_consensus.cc:2781] T 4722a1e5603a4fdf8081eb3ffaf52b92 P 25cbff182913473d8f139a262c007dcb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:37:17.242352 548 raft_consensus.cc:686] T 4722a1e5603a4fdf8081eb3ffaf52b92 P 25cbff182913473d8f139a262c007dcb [term 1 LEADER]: Becoming Leader. State: Replica: 25cbff182913473d8f139a262c007dcb, State: Running, Role: LEADER
I20260806 11:37:17.242748 548 consensus_queue.cc:237] T 4722a1e5603a4fdf8081eb3ffaf52b92 P 25cbff182913473d8f139a262c007dcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25cbff182913473d8f139a262c007dcb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38079 } } peers { permanent_uuid: "2ff841dd9de54f9693d49801f47524c5" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 36001 } } peers { permanent_uuid: "9aa4eeda4bc94ea296275e7e6ea65886" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 40799 } }
I20260806 11:37:17.246771 32494 catalog_manager.cc:5495] T 4722a1e5603a4fdf8081eb3ffaf52b92 P 25cbff182913473d8f139a262c007dcb reported cstate change: term changed from 0 to 1, leader changed from <none> to 25cbff182913473d8f139a262c007dcb (127.21.14.195). New cstate: current_term: 1 leader_uuid: "25cbff182913473d8f139a262c007dcb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25cbff182913473d8f139a262c007dcb" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 38079 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ff841dd9de54f9693d49801f47524c5" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 36001 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa4eeda4bc94ea296275e7e6ea65886" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 40799 } health_report { overall_health: UNKNOWN } } }
I20260806 11:37:17.298250 534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:37:17.293936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58432 (local address 127.21.14.195:38079)
0806 11:37:17.294098 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 11:37:17.294103 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:37:17.294120 (+ 17us) server_negotiation.cc:407] Connection header received
0806 11:37:17.294230 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:37:17.294233 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:37:17.294293 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:37:17.294429 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 11:37:17.294704 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:17.295269 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:17.296508 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:37:17.296759 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:37:17.296887 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:37:17.297251 (+ 364us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 11:37:17.297743 (+ 492us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 11:37:17.297891 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 11:37:17.297956 (+ 65us) server_negotiation.cc:299] Negotiation successful
0806 11:37:17.298046 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 11:37:17.305869 328 raft_consensus.cc:1261] T 4722a1e5603a4fdf8081eb3ffaf52b92 P 9aa4eeda4bc94ea296275e7e6ea65886 [term 1 FOLLOWER]: Refusing update from remote peer 25cbff182913473d8f139a262c007dcb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 11:37:17.305869 32650 raft_consensus.cc:1261] T 4722a1e5603a4fdf8081eb3ffaf52b92 P 2ff841dd9de54f9693d49801f47524c5 [term 1 FOLLOWER]: Refusing update from remote peer 25cbff182913473d8f139a262c007dcb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 11:37:17.306952 550 consensus_queue.cc:1035] T 4722a1e5603a4fdf8081eb3ffaf52b92 P 25cbff182913473d8f139a262c007dcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa4eeda4bc94ea296275e7e6ea65886" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 40799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 11:37:17.307219 548 consensus_queue.cc:1035] T 4722a1e5603a4fdf8081eb3ffaf52b92 P 25cbff182913473d8f139a262c007dcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ff841dd9de54f9693d49801f47524c5" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 36001 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 11:37:17.320478 554 mvcc.cc:205] Tried to move back new op lower bound from 7315522507994345472 to 7315522507749527552. Current Snapshot: MvccSnapshot[applied={T|T < 7315522507994345472}]
I20260806 11:37:17.327210 32494 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37640:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 11:37:17.327440 32494 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37640:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 11:37:17.330546 32494 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 87ec0dcbaddc409b8f4e8be753da01b2: Sending DeleteTablet for 3 replicas of tablet 4722a1e5603a4fdf8081eb3ffaf52b92
I20260806 11:37:17.331887 306 tablet_service.cc:1514] Processing DeleteTablet for tablet 4722a1e5603a4fdf8081eb3ffaf52b92 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:37:17 UTC) from {username='kudu', principal='kudu/127.21.14.254@KRBTEST.COM'} at 127.0.0.1:40496
I20260806 11:37:17.331882 451 tablet_service.cc:1514] Processing DeleteTablet for tablet 4722a1e5603a4fdf8081eb3ffaf52b92 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:37:17 UTC) from {username='kudu', principal='kudu/127.21.14.254@KRBTEST.COM'} at 127.0.0.1:58416
I20260806 11:37:17.331949 32630 tablet_service.cc:1514] Processing DeleteTablet for tablet 4722a1e5603a4fdf8081eb3ffaf52b92 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:37:17 UTC) from {username='kudu', principal='kudu/127.21.14.254@KRBTEST.COM'} at 127.0.0.1:56148
I20260806 11:37:17.332617 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 32562
I20260806 11:37:17.332608 566 tablet_replica.cc:330] T 4722a1e5603a4fdf8081eb3ffaf52b92 P 25cbff182913473d8f139a262c007dcb: stopping tablet replica
I20260806 11:37:17.333050 566 raft_consensus.cc:2229] T 4722a1e5603a4fdf8081eb3ffaf52b92 P 25cbff182913473d8f139a262c007dcb [term 1 LEADER]: Raft consensus shutting down.
I20260806 11:37:17.333555 566 raft_consensus.cc:2258] T 4722a1e5603a4fdf8081eb3ffaf52b92 P 25cbff182913473d8f139a262c007dcb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 11:37:17.334048 567 tablet_replica.cc:330] T 4722a1e5603a4fdf8081eb3ffaf52b92 P 9aa4eeda4bc94ea296275e7e6ea65886: stopping tablet replica
I20260806 11:37:17.334419 567 raft_consensus.cc:2229] T 4722a1e5603a4fdf8081eb3ffaf52b92 P 9aa4eeda4bc94ea296275e7e6ea65886 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 11:37:17.334793 567 raft_consensus.cc:2258] T 4722a1e5603a4fdf8081eb3ffaf52b92 P 9aa4eeda4bc94ea296275e7e6ea65886 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 11:37:17.336871 566 ts_tablet_manager.cc:1905] T 4722a1e5603a4fdf8081eb3ffaf52b92 P 25cbff182913473d8f139a262c007dcb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 11:37:17.336871 567 ts_tablet_manager.cc:1905] T 4722a1e5603a4fdf8081eb3ffaf52b92 P 9aa4eeda4bc94ea296275e7e6ea65886: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 11:37:17.340332 567 ts_tablet_manager.cc:1918] T 4722a1e5603a4fdf8081eb3ffaf52b92 P 9aa4eeda4bc94ea296275e7e6ea65886: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 11:37:17.340481 567 log.cc:1196] T 4722a1e5603a4fdf8081eb3ffaf52b92 P 9aa4eeda4bc94ea296275e7e6ea65886: Deleting WAL directory at /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/minicluster-data/ts-1/wal/wals/4722a1e5603a4fdf8081eb3ffaf52b92
I20260806 11:37:17.340771 566 ts_tablet_manager.cc:1918] T 4722a1e5603a4fdf8081eb3ffaf52b92 P 25cbff182913473d8f139a262c007dcb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 11:37:17.340842 567 ts_tablet_manager.cc:1939] T 4722a1e5603a4fdf8081eb3ffaf52b92 P 9aa4eeda4bc94ea296275e7e6ea65886: Deleting consensus metadata
I20260806 11:37:17.340905 566 log.cc:1196] T 4722a1e5603a4fdf8081eb3ffaf52b92 P 25cbff182913473d8f139a262c007dcb: Deleting WAL directory at /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786016169045816-21563-0/minicluster-data/ts-2/wal/wals/4722a1e5603a4fdf8081eb3ffaf52b92
I20260806 11:37:17.341305 566 ts_tablet_manager.cc:1939] T 4722a1e5603a4fdf8081eb3ffaf52b92 P 25cbff182913473d8f139a262c007dcb: Deleting consensus metadata
I20260806 11:37:17.343385 32477 catalog_manager.cc:4841] TS 9aa4eeda4bc94ea296275e7e6ea65886 (127.21.14.194:40799): tablet 4722a1e5603a4fdf8081eb3ffaf52b92 (table test-table [id=8b40630decde488f870d63bf5e5c8137]) successfully deleted
I20260806 11:37:17.343708 32481 catalog_manager.cc:4841] TS 25cbff182913473d8f139a262c007dcb (127.21.14.195:38079): tablet 4722a1e5603a4fdf8081eb3ffaf52b92 (table test-table [id=8b40630decde488f870d63bf5e5c8137]) successfully deleted
W20260806 11:37:17.345769 32481 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.14.193:36001 (error 108)
I20260806 11:37:17.346776 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 32708
I20260806 11:37:17.347088 533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:37:17.346726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.193:36001 (local address 127.0.0.1:56164)
0806 11:37:17.346887 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 11:37:17.346970 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.193:36001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260806 11:37:17.351612 32481 catalog_manager.cc:4584] TS 2ff841dd9de54f9693d49801f47524c5 (127.21.14.193:36001): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4722a1e5603a4fdf8081eb3ffaf52b92: Network error: Client connection negotiation failed: client connection to 127.21.14.193:36001: connect: Connection refused (error 111)
I20260806 11:37:17.357566 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 383
I20260806 11:37:17.366309 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 32461
2026-08-06T11:37:17Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2757 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 11:37:17 dist-test-slave-7c35 krb5kdc[572](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 11:37:17 dist-test-slave-7c35 krb5kdc[572](info): set up 2 sockets
Aug 06 11:37:17 dist-test-slave-7c35 krb5kdc[572](info): commencing operation
krb5kdc: starting...
W20260806 11:37:18.506351 21563 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.005s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Aug 06 11:37:18 dist-test-slave-7c35 krb5kdc[572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016238, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T11:37:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T11:37:18Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.21.14.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.21.14.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.21.14.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/krb5kdc/rangeradmin_127.21.14.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.21.14.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/krb5kdc/rangeradmin_127.21.14.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.21.14.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.21.14.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.21.14.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/krb5kdc/rangerlookup_127.21.14.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.21.14.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/krb5kdc/rangerlookup_127.21.14.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.21.14.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.21.14.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.21.14.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/krb5kdc/HTTP_127.21.14.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.21.14.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/krb5kdc/HTTP_127.21.14.212@KRBTEST.COM.keytab.
I20260806 11:37:18.740679 21563 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task4_zrTV/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/postgres -l logfile start
2026-08-06 11:37:22.550 UTC [604] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 11:37:22.550 UTC [604] LOG: listening on IPv4 address "127.21.14.212", port 43603
2026-08-06 11:37:22.556 UTC [604] LOG: listening on Unix socket "/tmp/.s.PGSQL.43603"
2026-08-06 11:37:22.596 UTC [607] LOG: database system was shut down at 2026-08-06 11:37:19 UTC
2026-08-06 11:37:22.603 UTC [604] LOG: database system is ready to accept connections
I20260806 11:37:23.587214 21563 mini_postgres.cc:96] Postgres bound to 43603
2026-08-06 11:37:23.596 UTC [615] FATAL: database "slave" does not exist
127.21.14.212:43603 - accepting connections
I20260806 11:37:23.597275 21563 mini_ranger.cc:161] Starting Ranger...
I20260806 11:37:23.619089 21563 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 11:37:24.305281 21563 mini_ranger.cc:87] Created ranger Postgres database
I20260806 11:37:24.305394 21563 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/ranger-admin
2026-08-06 11:37:24,659 [I] DB FLAVOR :POSTGRES
2026-08-06 11:37:24,660 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 11:37:24,660 [I] Checking connection..
2026-08-06 11:37:24,660 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4_zrTV/build/debug/bin/postgresql.jar:/tmp/dist-test-task4_zrTV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.14.212:43603/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 11:37:24,955 [I] Checking connection passed.
2026-08-06 11:37:24,955 [I] --------- Verifying version history table ---------
2026-08-06 11:37:24,955 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4_zrTV/build/debug/bin/postgresql.jar:/tmp/dist-test-task4_zrTV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.14.212:43603/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 11:37:25,248 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 11:37:25,249 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4_zrTV/build/debug/bin/postgresql.jar:/tmp/dist-test-task4_zrTV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.14.212:43603/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 11:37:25,520 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 11:37:25,520 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 11:37:25,520 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4_zrTV/build/debug/bin/postgresql.jar:/tmp/dist-test-task4_zrTV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.14.212:43603/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task4_zrTV/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 11:37:25.805 UTC [685] WARNING: there is no transaction in progress
2026-08-06 11:37:25,813 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 11:37:25,814 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4_zrTV/build/debug/bin/postgresql.jar:/tmp/dist-test-task4_zrTV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.14.212:43603/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 11:37:26,076 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 11:37:26,076 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 11:37:26,076 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4_zrTV/build/debug/bin/postgresql.jar:/tmp/dist-test-task4_zrTV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.14.212:43603/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 11:37:26,375 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4_zrTV/build/debug/bin/postgresql.jar:/tmp/dist-test-task4_zrTV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.14.212:43603/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 11:37:26,647 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4_zrTV/build/debug/bin/postgresql.jar:/tmp/dist-test-task4_zrTV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.14.212:43603/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-7c35.c.gcp-upstream.internal','N') ;"
2026-08-06 11:37:26,914 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4_zrTV/build/debug/bin/postgresql.jar:/tmp/dist-test-task4_zrTV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.14.212:43603/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 11:37:27,206 [I] Table x_portal_user does not exist in database ranger
2026-08-06 11:37:27,207 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4_zrTV/build/debug/bin/postgresql.jar:/tmp/dist-test-task4_zrTV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.14.212:43603/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 11:37:27,523 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 11:37:27,523 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4_zrTV/build/debug/bin/postgresql.jar:/tmp/dist-test-task4_zrTV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.14.212:43603/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 11:37:27,829 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4_zrTV/build/debug/bin/postgresql.jar:/tmp/dist-test-task4_zrTV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.14.212:43603/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 11:37:28,103 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 11:37:28,104 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4_zrTV/build/debug/bin/postgresql.jar:/tmp/dist-test-task4_zrTV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.14.212:43603/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task4_zrTV/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 11:37:29.398 UTC [801] WARNING: there is no transaction in progress
2026-08-06 11:37:29.424 UTC [801] WARNING: there is no transaction in progress
2026-08-06 11:37:29.448 UTC [801] WARNING: there is no transaction in progress
2026-08-06 11:37:29.472 UTC [801] WARNING: there is no transaction in progress
2026-08-06 11:37:29.497 UTC [801] WARNING: there is no transaction in progress
2026-08-06 11:37:29.525 UTC [801] WARNING: there is no transaction in progress
2026-08-06 11:37:29.606 UTC [801] WARNING: there is no transaction in progress
2026-08-06 11:37:29.618 UTC [801] WARNING: there is no transaction in progress
2026-08-06 11:37:29.643 UTC [801] WARNING: there is no transaction in progress
2026-08-06 11:37:29.656 UTC [801] WARNING: there is no transaction in progress
2026-08-06 11:37:29.671 UTC [801] WARNING: there is no transaction in progress
2026-08-06 11:37:29.691 UTC [801] WARNING: there is no transaction in progress
2026-08-06 11:37:29.704 UTC [801] WARNING: there is no transaction in progress
2026-08-06 11:37:29.713 UTC [801] WARNING: there is no transaction in progress
2026-08-06 11:37:29.726 UTC [801] WARNING: there is no transaction in progress
2026-08-06 11:37:29.739 UTC [801] WARNING: there is no transaction in progress
2026-08-06 11:37:30,838 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 11:37:30,839 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4_zrTV/build/debug/bin/postgresql.jar:/tmp/dist-test-task4_zrTV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.14.212:43603/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 11:37:31,088 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 11:37:31,088 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4_zrTV/build/debug/bin/postgresql.jar:/tmp/dist-test-task4_zrTV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.14.212:43603/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 11:37:31,371 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 11:37:31,371 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4_zrTV/build/debug/bin/postgresql.jar:/tmp/dist-test-task4_zrTV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.14.212:43603/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 11:37:31,674 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 11:37:31,675 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4_zrTV/build/debug/bin/postgresql.jar:/tmp/dist-test-task4_zrTV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.14.212:43603/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 11:37:31,944 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4_zrTV/build/debug/bin/postgresql.jar:/tmp/dist-test-task4_zrTV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.14.212:43603/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 11:37:32,226 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4_zrTV/build/debug/bin/postgresql.jar:/tmp/dist-test-task4_zrTV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.14.212:43603/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-7c35.c.gcp-upstream.internal';"
2026-08-06 11:37:32,503 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 11:37:32,504 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4_zrTV/build/debug/bin/postgresql.jar:/tmp/dist-test-task4_zrTV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.14.212:43603/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 11:37:32,795 [I] DB_PATCHES have already been applied
I20260806 11:37:32.801826 21563 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/ranger-admin:/tmp/dist-test-task4_zrTV/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task4_zrTV/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task4_zrTV/build/debug/bin/postgresql.jar:/tmp/dist-test-task4_zrTV/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 11:37:32.801944 21563 mini_ranger.cc:193] Using host: 127.21.14.212
I20260806 11:37:32.807318 21563 mini_ranger.cc:227] Ranger admin URL: http://127.21.14.212:37313
Aug 06, 2026 11:37:33 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 11:37:33 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 11:37:33 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 11:37:33 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 11:37:33 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 11:37:33 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 11:37:33 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 11:37:34.018260 21563 mini_ranger.cc:160] Time spent starting Ranger: real 10.421s user 0.003s sys 0.013s
Aug 06, 2026 11:37:56 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 11:37:57 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 06, 2026 11:37:57 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 11:37:57 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 11:37:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 11:37:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 11:37:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 11:37:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 11:37:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 11:37:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 11:37:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 11:37:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 11:37:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 11:37:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 11:37:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 11:37:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 11:37:57 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 11:37:58 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 11:37:58.837881 21563 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.21.14.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.21.14.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.21.14.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/krb5kdc/rangerkms_127.21.14.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.21.14.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/krb5kdc/rangerkms_127.21.14.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.21.14.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.21.14.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.21.14.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/krb5kdc/HTTP_127.21.14.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.21.14.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/krb5kdc/HTTP_127.21.14.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 11:37:58 dist-test-slave-7c35 krb5kdc[572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016278, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 11:37:58.942996 21563 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 11:37:58.958355 21563 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 11:38:00.337493 21563 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 11:38:00.337615 21563 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/ranger-kms
I20260806 11:38:00.337729 21563 mini_ranger_kms.cc:227] Using postgres at 127.21.14.212:43603
2026-08-06 11:38:01,201 [I] DB FLAVOR :POSTGRES
2026-08-06 11:38:01,201 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 11:38:01,201 [I] Checking connection
2026-08-06 11:38:01,201 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4_zrTV/build/debug/bin/postgresql.jar:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.14.212:43603/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 11:38:01,658 [I] connection success
2026-08-06 11:38:01,658 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 11:38:01,658 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 11:38:01,658 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4_zrTV/build/debug/bin/postgresql.jar:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.14.212:43603/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 11:38:01,957 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 11:38:01,957 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 11:38:01,957 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 11:38:01,957 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4_zrTV/build/debug/bin/postgresql.jar:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.14.212:43603/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 11:38:02,298 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 11:38:02.870913 21563 mini_ranger_kms.cc:310] Created kms service
I20260806 11:38:03.085848 21563 mini_ranger_kms.cc:326] Created kudu user
I20260806 11:38:03.152938 21563 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 11:38:03 dist-test-slave-7c35 krb5kdc[572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016278, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.21.14.212@KRBTEST.COM
I20260806 11:38:03.596791 21563 mini_ranger_kms.cc:384] Added ranger policy
I20260806 11:38:03.597008 21563 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/ranger-kms:/tmp/dist-test-task4_zrTV/build/debug/bin/postgresql.jar:/tmp/dist-test-task4_zrTV/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task4_zrTV/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 11:38:03.597039 21563 mini_ranger_kms.cc:240] Using host: 127.21.14.212
I20260806 11:38:03.602173 21563 mini_ranger_kms.cc:276] Ranger KMS PID: 993
I20260806 11:38:03.602360 21563 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.21.14.212:40975
Aug 06, 2026 11:38:05 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 11:38:05 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 11:38:05 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 11:38:05 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 11:38:05 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 11:38:05 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260806 11:38:06.068583 21563 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 7.126s user 0.000s sys 0.521s
I20260806 11:38:06.068739 21563 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 11:38:06.068892 21563 mini_ranger_kms.cc:401] 127.21.14.212:40975/kms/v1/keys
Aug 06 11:38:06 dist-test-slave-7c35 krb5kdc[572](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786016286, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.14.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 06 11:38:07 dist-test-slave-7c35 krb5kdc[572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016286, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.14.212@KRBTEST.COM for HTTP/127.21.14.212@KRBTEST.COM
Aug 06 11:38:07 dist-test-slave-7c35 krb5kdc[572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016286, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.14.212@KRBTEST.COM for HTTP/127.21.14.212@KRBTEST.COM
Aug 06 11:38:10 dist-test-slave-7c35 krb5kdc[572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016290, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.21.14.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab.
Entry for principal kudu/127.21.14.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab.
I20260806 11:38:10.596186 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:44873
--webserver_interface=127.21.14.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab
--principal=kudu/127.21.14.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:36329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.14.254:44873
--ranger_config_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.14.212:40975/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:38:10.758044 1046 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:38:10.758407 1046 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:38:10.758497 1046 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:38:10.763701 1046 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 11:38:10.763813 1046 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:38:10.763834 1046 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:38:10.763852 1046 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 11:38:10.763868 1046 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 11:38:10.771370 1046 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:36329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/master-0/wal
--ranger_kms_url=127.21.14.212:40975/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.21.14.254:44873
--ranger_config_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab
--principal=kudu/127.21.14.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:44873
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.14.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1046
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:38:10.772665 1046 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:38:10.773849 1046 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:38:10.784832 1053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:38:10.785094 1052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:38:10.785209 1046 server_base.cc:1034] running on GCE node
W20260806 11:38:10.785450 1055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:38:10.786374 1046 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:38:10.787817 1046 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:38:10.789053 1046 hybrid_clock.cc:648] HybridClock initialized: now 1786016290789019 us; error 56 us; skew 500 ppm
Aug 06 11:38:10 dist-test-slave-7c35 krb5kdc[572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016290, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:38:10.793217 1046 init.cc:377] Logged in from keytab as kudu/127.21.14.254@KRBTEST.COM (short username kudu)
I20260806 11:38:10.794497 1046 webserver.cc:460] Webserver started at http://127.21.14.254:43909/ using document root <none> and password file <none>
I20260806 11:38:10.795226 1046 fs_manager.cc:359] Metadata directory not provided
I20260806 11:38:10.795303 1046 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:38:10.795558 1046 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 11:38:10 dist-test-slave-7c35 krb5kdc[572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016290, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.254@KRBTEST.COM for HTTP/127.21.14.212@KRBTEST.COM
I20260806 11:38:10.866739 1046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/master-0/data/instance:
uuid: "9da2d9c342494b0da1108585a2c826d4"
format_stamp: "Formatted at 2026-08-06 11:38:10 on dist-test-slave-7c35"
server_key: "e39c5b99f8c989bb2332fd1c1cf98a72"
server_key_iv: "524c26a65c2e4d797bb8cedab5378447"
server_key_version: "kuduclusterkey@0"
I20260806 11:38:10.867604 1046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/master-0/wal/instance:
uuid: "9da2d9c342494b0da1108585a2c826d4"
format_stamp: "Formatted at 2026-08-06 11:38:10 on dist-test-slave-7c35"
server_key: "e39c5b99f8c989bb2332fd1c1cf98a72"
server_key_iv: "524c26a65c2e4d797bb8cedab5378447"
server_key_version: "kuduclusterkey@0"
I20260806 11:38:10.872344 1046 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 11:38:10.891510 1062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:38:10.893121 1046 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260806 11:38:10.893343 1046 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/master-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/master-0/wal
uuid: "9da2d9c342494b0da1108585a2c826d4"
format_stamp: "Formatted at 2026-08-06 11:38:10 on dist-test-slave-7c35"
server_key: "e39c5b99f8c989bb2332fd1c1cf98a72"
server_key_iv: "524c26a65c2e4d797bb8cedab5378447"
server_key_version: "kuduclusterkey@0"
I20260806 11:38:10.893507 1046 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:38:10.913869 1046 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:38:10.917626 1046 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:38:10.917930 1046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:38:10.936467 1046 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.254:44873
I20260806 11:38:10.936647 1118 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.254:44873 every 8 connection(s)
I20260806 11:38:10.937743 1046 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
I20260806 11:38:10.938120 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 1046
I20260806 11:38:10.938256 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/master-0/wal/instance
I20260806 11:38:10.938596 21563 external_mini_cluster.cc:1428] Setting key c9b671b3d2e3a3910918d73636d3a058
I20260806 11:38:10.942466 1120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:38:10.951332 1120 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9da2d9c342494b0da1108585a2c826d4: Bootstrap starting.
Aug 06 11:38:10 dist-test-slave-7c35 krb5kdc[572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016290, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:38:10.955191 1120 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9da2d9c342494b0da1108585a2c826d4: Neither blocks nor log segments found. Creating new log.
I20260806 11:38:10.957751 1120 log.cc:824] T 00000000000000000000000000000000 P 9da2d9c342494b0da1108585a2c826d4: Log is configured to *not* fsync() on all Append() calls
I20260806 11:38:10.961045 1120 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9da2d9c342494b0da1108585a2c826d4: No bootstrap required, opened a new log
I20260806 11:38:10.962689 1123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:38:10.940436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38644 (local address 127.21.14.254:44873)
0806 11:38:10.943721 (+ 3285us) server_negotiation.cc:206] Beginning negotiation
0806 11:38:10.943777 (+ 56us) server_negotiation.cc:399] Waiting for connection header
0806 11:38:10.943827 (+ 50us) server_negotiation.cc:407] Connection header received
0806 11:38:10.945161 (+ 1334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:38:10.945191 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:38:10.945691 (+ 500us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:38:10.946145 (+ 454us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:38:10.947388 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:38:10.949287 (+ 1899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:38:10.950586 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:38:10.951440 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:38:10.951620 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:38:10.955471 (+ 3851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:38:10.955483 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:38:10.955508 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:38:10.955551 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:38:10.958378 (+ 2827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:38:10.958993 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:38:10.959001 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:38:10.959009 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:38:10.959210 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:38:10.959595 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:38:10.959599 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:38:10.959600 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:38:10.960181 (+ 581us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:38:10.960404 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0806 11:38:10.960753 (+ 349us) server_negotiation.cc:299] Negotiation successful
0806 11:38:10.961270 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3105,"thread_start_us":324,"threads_started":1}
I20260806 11:38:10.966089 1120 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9da2d9c342494b0da1108585a2c826d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9da2d9c342494b0da1108585a2c826d4" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 44873 } }
I20260806 11:38:10.966346 1120 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9da2d9c342494b0da1108585a2c826d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:38:10.966384 1120 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9da2d9c342494b0da1108585a2c826d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9da2d9c342494b0da1108585a2c826d4, State: Initialized, Role: FOLLOWER
I20260806 11:38:10.966987 1120 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9da2d9c342494b0da1108585a2c826d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9da2d9c342494b0da1108585a2c826d4" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 44873 } }
I20260806 11:38:10.967141 1120 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9da2d9c342494b0da1108585a2c826d4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 11:38:10.967228 1120 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9da2d9c342494b0da1108585a2c826d4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 11:38:10.967334 1120 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9da2d9c342494b0da1108585a2c826d4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:38:10.968608 1120 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9da2d9c342494b0da1108585a2c826d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9da2d9c342494b0da1108585a2c826d4" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 44873 } }
I20260806 11:38:10.969010 1120 leader_election.cc:302] T 00000000000000000000000000000000 P 9da2d9c342494b0da1108585a2c826d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9da2d9c342494b0da1108585a2c826d4; no voters:
I20260806 11:38:10.969437 1120 leader_election.cc:288] T 00000000000000000000000000000000 P 9da2d9c342494b0da1108585a2c826d4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 11:38:10.969705 1125 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9da2d9c342494b0da1108585a2c826d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:38:10.970214 1125 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9da2d9c342494b0da1108585a2c826d4 [term 1 LEADER]: Becoming Leader. State: Replica: 9da2d9c342494b0da1108585a2c826d4, State: Running, Role: LEADER
I20260806 11:38:10.970626 1125 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9da2d9c342494b0da1108585a2c826d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9da2d9c342494b0da1108585a2c826d4" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 44873 } }
I20260806 11:38:10.971222 1120 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9da2d9c342494b0da1108585a2c826d4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 11:38:10.972118 1125 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9da2d9c342494b0da1108585a2c826d4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9da2d9c342494b0da1108585a2c826d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9da2d9c342494b0da1108585a2c826d4" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 44873 } } }
I20260806 11:38:10.972424 1125 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9da2d9c342494b0da1108585a2c826d4 [sys.catalog]: This master's current role is: LEADER
I20260806 11:38:10.972796 1125 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9da2d9c342494b0da1108585a2c826d4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9da2d9c342494b0da1108585a2c826d4. Latest consensus state: current_term: 1 leader_uuid: "9da2d9c342494b0da1108585a2c826d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9da2d9c342494b0da1108585a2c826d4" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 44873 } } }
I20260806 11:38:10.972959 1125 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9da2d9c342494b0da1108585a2c826d4 [sys.catalog]: This master's current role is: LEADER
I20260806 11:38:10.974584 1120 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 11:38:10.975556 1134 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 11:38:10.986539 1134 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 11:38:10.996950 1134 catalog_manager.cc:1347] Generated new cluster ID: 97030850b79642d3976b53be35c55478
I20260806 11:38:10.997160 1134 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 11:38:11.007310 1134 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 11:38:11.008968 1134 catalog_manager.cc:1504] Loading token signing keys...
I20260806 11:38:11.033245 1134 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9da2d9c342494b0da1108585a2c826d4: Generated new TSK 0
I20260806 11:38:11.034565 1134 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.14.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab.
Entry for principal kudu/127.21.14.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab.
I20260806 11:38:12.075500 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.14.212:40975/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.193:0
--local_ip_for_outbound_sockets=127.21.14.193
--webserver_interface=127.21.14.193
--webserver_port=0
--tserver_master_addrs=127.21.14.254:44873
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab
--principal=kudu/127.21.14.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:36329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:38:12.272627 1169 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:38:12.273016 1169 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:38:12.273123 1169 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:38:12.277850 1169 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:38:12.277954 1169 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:38:12.278061 1169 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.193
I20260806 11:38:12.285068 1169 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:36329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.21.14.212:40975/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab
--principal=kudu/127.21.14.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.14.193
--webserver_port=0
--tserver_master_addrs=127.21.14.254:44873
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1169
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.193
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:38:12.286432 1169 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:38:12.287868 1169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:38:12.301981 1175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:38:12.302016 1179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:38:12.302034 1176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:38:12.303565 1169 server_base.cc:1034] running on GCE node
I20260806 11:38:12.304100 1169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:38:12.304922 1169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:38:12.306171 1169 hybrid_clock.cc:648] HybridClock initialized: now 1786016292306124 us; error 69 us; skew 500 ppm
Aug 06 11:38:12 dist-test-slave-7c35 krb5kdc[572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016292, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:38:12.310230 1169 init.cc:377] Logged in from keytab as kudu/127.21.14.193@KRBTEST.COM (short username kudu)
I20260806 11:38:12.311533 1169 webserver.cc:460] Webserver started at http://127.21.14.193:40151/ using document root <none> and password file <none>
I20260806 11:38:12.312191 1169 fs_manager.cc:359] Metadata directory not provided
I20260806 11:38:12.312290 1169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:38:12.312536 1169 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 11:38:12 dist-test-slave-7c35 krb5kdc[572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016292, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.193@KRBTEST.COM for HTTP/127.21.14.212@KRBTEST.COM
I20260806 11:38:12.330673 1169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/ts-0/data/instance:
uuid: "d4b0211e0d17418e8360cc9ba778b3f3"
format_stamp: "Formatted at 2026-08-06 11:38:12 on dist-test-slave-7c35"
server_key: "ff8a90d3e8498966f367f27a37a287f9"
server_key_iv: "c4596a6ec21178506e05e304286f457d"
server_key_version: "kuduclusterkey@0"
I20260806 11:38:12.331756 1169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/ts-0/wal/instance:
uuid: "d4b0211e0d17418e8360cc9ba778b3f3"
format_stamp: "Formatted at 2026-08-06 11:38:12 on dist-test-slave-7c35"
server_key: "ff8a90d3e8498966f367f27a37a287f9"
server_key_iv: "c4596a6ec21178506e05e304286f457d"
server_key_version: "kuduclusterkey@0"
I20260806 11:38:12.337850 1169 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260806 11:38:12.359380 1189 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:38:12.361019 1169 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 11:38:12.361219 1169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/ts-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/ts-0/wal
uuid: "d4b0211e0d17418e8360cc9ba778b3f3"
format_stamp: "Formatted at 2026-08-06 11:38:12 on dist-test-slave-7c35"
server_key: "ff8a90d3e8498966f367f27a37a287f9"
server_key_iv: "c4596a6ec21178506e05e304286f457d"
server_key_version: "kuduclusterkey@0"
I20260806 11:38:12.361375 1169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:38:12.381706 1169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:38:12.385437 1169 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:38:12.385741 1169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:38:12.386620 1169 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:38:12.387867 1169 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:38:12.387933 1169 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:38:12.387990 1169 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:38:12.388012 1169 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:38:12.416831 1169 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.193:41713
I20260806 11:38:12.416970 1319 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.193:41713 every 8 connection(s)
I20260806 11:38:12.418448 1169 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
I20260806 11:38:12.421537 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 1169
I20260806 11:38:12.421696 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/ts-0/wal/instance
I20260806 11:38:12.422108 21563 external_mini_cluster.cc:1428] Setting key d5a0baf9c263a34cd94dd8501d88add3
Aug 06 11:38:12 dist-test-slave-7c35 krb5kdc[572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016292, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.193@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.14.194@KRBTEST.COM; defaulting to no policy
I20260806 11:38:12.441143 1323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:38:12.421169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:47951 (local address 127.21.14.254:44873)
0806 11:38:12.421546 (+ 377us) server_negotiation.cc:206] Beginning negotiation
0806 11:38:12.421551 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:38:12.422719 (+ 1168us) server_negotiation.cc:407] Connection header received
0806 11:38:12.424336 (+ 1617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:38:12.424342 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:38:12.424403 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:38:12.424475 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:38:12.426257 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:38:12.426950 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:38:12.429215 (+ 2265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:38:12.429718 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:38:12.429893 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:38:12.434190 (+ 4297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:38:12.434198 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:38:12.434203 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:38:12.434250 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:38:12.437064 (+ 2814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:38:12.438117 (+ 1053us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:38:12.438124 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:38:12.438126 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:38:12.438280 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:38:12.438929 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:38:12.438935 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:38:12.438937 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:38:12.439274 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:38:12.439404 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 11:38:12.440561 (+ 1157us) server_negotiation.cc:299] Negotiation successful
0806 11:38:12.440755 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":150,"threads_started":1}
I20260806 11:38:12.442767 1324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:38:12.421720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:44873 (local address 127.21.14.193:47951)
0806 11:38:12.422522 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0806 11:38:12.422588 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0806 11:38:12.424057 (+ 1469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:38:12.424858 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:38:12.424866 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:38:12.425341 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:38:12.426072 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:38:12.426091 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:38:12.427189 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:38:12.427195 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:38:12.428987 (+ 1792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:38:12.429019 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:38:12.429951 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:38:12.429956 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:38:12.430219 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:38:12.431090 (+ 871us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:38:12.431120 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:38:12.433850 (+ 2730us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:38:12.437286 (+ 3436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:38:12.437305 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:38:12.437341 (+ 36us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:38:12.437922 (+ 581us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:38:12.438468 (+ 546us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:38:12.438472 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:38:12.438474 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:38:12.438757 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:38:12.439432 (+ 675us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:38:12.439441 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:38:12.439955 (+ 514us) client_negotiation.cc:769] Sending connection context
0806 11:38:12.440245 (+ 290us) client_negotiation.cc:240] Negotiation successful
0806 11:38:12.440834 (+ 589us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":295,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.194@KRBTEST.COM" created.
I20260806 11:38:12.447683 1320 heartbeater.cc:347] Connected to a master server at 127.21.14.254:44873
I20260806 11:38:12.448056 1320 heartbeater.cc:464] Registering TS with master...
I20260806 11:38:12.448808 1320 heartbeater.cc:511] Master 127.21.14.254:44873 requested a full tablet report, sending...
I20260806 11:38:12.451180 1081 ts_manager.cc:194] Registered new tserver with Master: d4b0211e0d17418e8360cc9ba778b3f3 (127.21.14.193:41713)
I20260806 11:38:12.452797 1081 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.14.193@KRBTEST.COM'} at 127.21.14.193:47951
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab.
Entry for principal kudu/127.21.14.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab.
I20260806 11:38:12.470928 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.14.212:40975/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.194:0
--local_ip_for_outbound_sockets=127.21.14.194
--webserver_interface=127.21.14.194
--webserver_port=0
--tserver_master_addrs=127.21.14.254:44873
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab
--principal=kudu/127.21.14.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:36329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 11:38:12 dist-test-slave-7c35 krb5kdc[572](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786016292, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 11:38:12.702265 1327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:38:12.702685 1327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:38:12.702782 1327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:38:12.708436 1327 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:38:12.708523 1327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:38:12.708647 1327 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.194
I20260806 11:38:12.717289 1327 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:36329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.21.14.212:40975/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab
--principal=kudu/127.21.14.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.14.194
--webserver_port=0
--tserver_master_addrs=127.21.14.254:44873
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1327
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.194
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:38:12.718818 1327 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:38:12.720305 1327 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:38:12.735164 1339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:38:12.735275 1337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:38:12.736083 1336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:38:12.736872 1327 server_base.cc:1034] running on GCE node
I20260806 11:38:12.737502 1327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:38:12.738551 1327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:38:12.739780 1327 hybrid_clock.cc:648] HybridClock initialized: now 1786016292739714 us; error 109 us; skew 500 ppm
Aug 06 11:38:12 dist-test-slave-7c35 krb5kdc[572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016292, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:38:12.745267 1327 init.cc:377] Logged in from keytab as kudu/127.21.14.194@KRBTEST.COM (short username kudu)
I20260806 11:38:12.746589 1327 webserver.cc:460] Webserver started at http://127.21.14.194:40079/ using document root <none> and password file <none>
I20260806 11:38:12.747574 1327 fs_manager.cc:359] Metadata directory not provided
I20260806 11:38:12.747706 1327 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:38:12.748039 1327 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 11:38:12 dist-test-slave-7c35 krb5kdc[572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016292, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.194@KRBTEST.COM for HTTP/127.21.14.212@KRBTEST.COM
I20260806 11:38:12.767663 1327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/ts-1/data/instance:
uuid: "3af5d13ab5dd482a9b5926b8bdbcbbea"
format_stamp: "Formatted at 2026-08-06 11:38:12 on dist-test-slave-7c35"
server_key: "703823647ad163880e4d51baf5a88687"
server_key_iv: "bc2dbba8e6c3d320faf2126c94c1282d"
server_key_version: "kuduclusterkey@0"
I20260806 11:38:12.768663 1327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/ts-1/wal/instance:
uuid: "3af5d13ab5dd482a9b5926b8bdbcbbea"
format_stamp: "Formatted at 2026-08-06 11:38:12 on dist-test-slave-7c35"
server_key: "703823647ad163880e4d51baf5a88687"
server_key_iv: "bc2dbba8e6c3d320faf2126c94c1282d"
server_key_version: "kuduclusterkey@0"
I20260806 11:38:12.777985 1327 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.005s sys 0.000s
I20260806 11:38:12.803469 1346 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:38:12.805573 1327 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 11:38:12.805766 1327 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/ts-1/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/ts-1/wal
uuid: "3af5d13ab5dd482a9b5926b8bdbcbbea"
format_stamp: "Formatted at 2026-08-06 11:38:12 on dist-test-slave-7c35"
server_key: "703823647ad163880e4d51baf5a88687"
server_key_iv: "bc2dbba8e6c3d320faf2126c94c1282d"
server_key_version: "kuduclusterkey@0"
I20260806 11:38:12.805938 1327 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:38:12.835999 1327 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:38:12.839871 1327 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:38:12.840129 1327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:38:12.840917 1327 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:38:12.842013 1327 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:38:12.842092 1327 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:38:12.842165 1327 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:38:12.842219 1327 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:38:12.861527 1327 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.194:37165
I20260806 11:38:12.862171 1468 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.194:37165 every 8 connection(s)
I20260806 11:38:12.863423 1327 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
I20260806 11:38:12.868170 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 1327
I20260806 11:38:12.868331 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/ts-1/wal/instance
I20260806 11:38:12.868673 21563 external_mini_cluster.cc:1428] Setting key 5a12094e50fb49a224677b90df82acad
Aug 06 11:38:12 dist-test-slave-7c35 krb5kdc[572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016292, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.194@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:38:12.886418 1323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:38:12.870016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:57735 (local address 127.21.14.254:44873)
0806 11:38:12.870168 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 11:38:12.870174 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 11:38:12.870609 (+ 435us) server_negotiation.cc:407] Connection header received
0806 11:38:12.872190 (+ 1581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:38:12.872193 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:38:12.872246 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:38:12.872300 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:38:12.874266 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:38:12.875153 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:38:12.877036 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:38:12.877399 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:38:12.877492 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:38:12.881411 (+ 3919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:38:12.881415 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:38:12.881418 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:38:12.881452 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:38:12.883483 (+ 2031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:38:12.884257 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:38:12.884261 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:38:12.884263 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:38:12.884360 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:38:12.884837 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:38:12.884841 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:38:12.884843 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:38:12.885097 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:38:12.885177 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 11:38:12.886070 (+ 893us) server_negotiation.cc:299] Negotiation successful
0806 11:38:12.886236 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
WARNING: no policy specified for kudu/127.21.14.195@KRBTEST.COM; defaulting to no policy
I20260806 11:38:12.892338 1471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:38:12.868766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:44873 (local address 127.21.14.194:57735)
0806 11:38:12.870439 (+ 1673us) negotiation.cc:107] Waiting for socket to connect
0806 11:38:12.870502 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0806 11:38:12.871919 (+ 1417us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:38:12.872756 (+ 837us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:38:12.872767 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:38:12.873244 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:38:12.874062 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:38:12.874087 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:38:12.875341 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:38:12.875344 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:38:12.876862 (+ 1518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:38:12.876875 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:38:12.877734 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:38:12.877738 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:38:12.877999 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:38:12.878819 (+ 820us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:38:12.878871 (+ 52us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:38:12.881138 (+ 2267us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:38:12.883674 (+ 2536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:38:12.883683 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:38:12.883705 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:38:12.884116 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:38:12.884483 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:38:12.884486 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:38:12.884488 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:38:12.884710 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:38:12.885266 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:38:12.885274 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:38:12.885685 (+ 411us) client_negotiation.cc:769] Sending connection context
0806 11:38:12.887193 (+ 1508us) client_negotiation.cc:240] Negotiation successful
0806 11:38:12.887676 (+ 483us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1425,"thread_start_us":194,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.195@KRBTEST.COM" created.
I20260806 11:38:12.894706 1469 heartbeater.cc:347] Connected to a master server at 127.21.14.254:44873
I20260806 11:38:12.895046 1469 heartbeater.cc:464] Registering TS with master...
I20260806 11:38:12.895802 1469 heartbeater.cc:511] Master 127.21.14.254:44873 requested a full tablet report, sending...
I20260806 11:38:12.897492 1081 ts_manager.cc:194] Registered new tserver with Master: 3af5d13ab5dd482a9b5926b8bdbcbbea (127.21.14.194:37165)
I20260806 11:38:12.898365 1081 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.14.194@KRBTEST.COM'} at 127.21.14.194:57735
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab.
Entry for principal kudu/127.21.14.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab.
I20260806 11:38:12.924412 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.14.212:40975/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.195:0
--local_ip_for_outbound_sockets=127.21.14.195
--webserver_interface=127.21.14.195
--webserver_port=0
--tserver_master_addrs=127.21.14.254:44873
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab
--principal=kudu/127.21.14.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:36329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:38:13.324517 1474 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:38:13.325397 1474 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:38:13.325621 1474 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:38:13.337052 1474 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:38:13.337316 1474 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:38:13.337594 1474 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.195
I20260806 11:38:13.351009 1474 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:36329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.21.14.212:40975/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab
--principal=kudu/127.21.14.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.14.195
--webserver_port=0
--tserver_master_addrs=127.21.14.254:44873
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1474
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.195
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:38:13.354123 1474 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:38:13.356699 1474 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:38:13.389206 1479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:38:13.389573 1480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:38:13.391680 1482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:38:13.393352 1474 server_base.cc:1034] running on GCE node
I20260806 11:38:13.394613 1474 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:38:13.396432 1474 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:38:13.400086 1474 hybrid_clock.cc:648] HybridClock initialized: now 1786016293399949 us; error 171 us; skew 500 ppm
Aug 06 11:38:13 dist-test-slave-7c35 krb5kdc[572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016293, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:38:13.411440 1474 init.cc:377] Logged in from keytab as kudu/127.21.14.195@KRBTEST.COM (short username kudu)
I20260806 11:38:13.414594 1474 webserver.cc:460] Webserver started at http://127.21.14.195:43427/ using document root <none> and password file <none>
I20260806 11:38:13.416575 1474 fs_manager.cc:359] Metadata directory not provided
I20260806 11:38:13.416955 1474 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:38:13.417635 1474 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 11:38:13 dist-test-slave-7c35 krb5kdc[572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016293, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.195@KRBTEST.COM for HTTP/127.21.14.212@KRBTEST.COM
I20260806 11:38:13.459214 1320 heartbeater.cc:503] Master 127.21.14.254:44873 was elected leader, sending a full tablet report...
I20260806 11:38:13.475606 1474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/ts-2/data/instance:
uuid: "1c01366def2a4081b18751497a17cb31"
format_stamp: "Formatted at 2026-08-06 11:38:13 on dist-test-slave-7c35"
server_key: "98564965f19d822c9c1b4e2b379a8ad6"
server_key_iv: "3f2f3123d9c47adf67efb65a536fcca5"
server_key_version: "kuduclusterkey@0"
I20260806 11:38:13.477792 1474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/ts-2/wal/instance:
uuid: "1c01366def2a4081b18751497a17cb31"
format_stamp: "Formatted at 2026-08-06 11:38:13 on dist-test-slave-7c35"
server_key: "98564965f19d822c9c1b4e2b379a8ad6"
server_key_iv: "3f2f3123d9c47adf67efb65a536fcca5"
server_key_version: "kuduclusterkey@0"
I20260806 11:38:13.492250 1474 fs_manager.cc:691] Time spent creating directory manager: real 0.012s user 0.008s sys 0.000s
I20260806 11:38:13.535770 1490 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:38:13.539109 1474 fs_manager.cc:725] Time spent opening block manager: real 0.013s user 0.007s sys 0.000s
I20260806 11:38:13.539492 1474 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/ts-2/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/ts-2/wal
uuid: "1c01366def2a4081b18751497a17cb31"
format_stamp: "Formatted at 2026-08-06 11:38:13 on dist-test-slave-7c35"
server_key: "98564965f19d822c9c1b4e2b379a8ad6"
server_key_iv: "3f2f3123d9c47adf67efb65a536fcca5"
server_key_version: "kuduclusterkey@0"
I20260806 11:38:13.539971 1474 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:38:13.569108 1474 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:38:13.576895 1474 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:38:13.577680 1474 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:38:13.579653 1474 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:38:13.582273 1474 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:38:13.582486 1474 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260806 11:38:13.582708 1474 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:38:13.582834 1474 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:38:13.638216 1474 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.195:45989
I20260806 11:38:13.641053 1474 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
I20260806 11:38:13.644899 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 1474
I20260806 11:38:13.645107 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/ts-2/wal/instance
I20260806 11:38:13.645804 21563 external_mini_cluster.cc:1428] Setting key b27c634fdbb7a806b63164011db0a0fc
I20260806 11:38:13.644316 1620 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.195:45989 every 8 connection(s)
Aug 06 11:38:13 dist-test-slave-7c35 krb5kdc[572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016293, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.195@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:38:13.685214 1625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:38:13.652637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:35743 (local address 127.21.14.254:44873)
0806 11:38:13.655391 (+ 2754us) server_negotiation.cc:206] Beginning negotiation
0806 11:38:13.655409 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 11:38:13.659120 (+ 3711us) server_negotiation.cc:407] Connection header received
0806 11:38:13.659215 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:38:13.659218 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:38:13.659344 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:38:13.659535 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:38:13.663917 (+ 4382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:38:13.665246 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:38:13.669120 (+ 3874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:38:13.669778 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:38:13.669987 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:38:13.676906 (+ 6919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:38:13.676914 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:38:13.676916 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:38:13.676991 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:38:13.680412 (+ 3421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:38:13.681614 (+ 1202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:38:13.681618 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:38:13.681620 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:38:13.681761 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:38:13.682552 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:38:13.682564 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:38:13.682567 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:38:13.683127 (+ 560us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:38:13.683346 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0806 11:38:13.684625 (+ 1279us) server_negotiation.cc:299] Negotiation successful
0806 11:38:13.684893 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2498,"thread_start_us":223,"threads_started":1}
I20260806 11:38:13.688755 1623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:38:13.650864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:44873 (local address 127.21.14.195:35743)
0806 11:38:13.656036 (+ 5172us) negotiation.cc:107] Waiting for socket to connect
0806 11:38:13.656158 (+ 122us) client_negotiation.cc:174] Beginning negotiation
0806 11:38:13.658403 (+ 2245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:38:13.660235 (+ 1832us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:38:13.660267 (+ 32us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:38:13.661568 (+ 1301us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:38:13.663422 (+ 1854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:38:13.663479 (+ 57us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:38:13.665615 (+ 2136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:38:13.665631 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:38:13.668769 (+ 3138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:38:13.668799 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:38:13.670498 (+ 1699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:38:13.670504 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:38:13.671129 (+ 625us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:38:13.672743 (+ 1614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:38:13.672772 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:38:13.676477 (+ 3705us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:38:13.680697 (+ 4220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:38:13.680707 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:38:13.680748 (+ 41us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:38:13.681408 (+ 660us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:38:13.681962 (+ 554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:38:13.681968 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:38:13.681970 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:38:13.682356 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:38:13.683471 (+ 1115us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:38:13.683480 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:38:13.684112 (+ 632us) client_negotiation.cc:769] Sending connection context
0806 11:38:13.686058 (+ 1946us) client_negotiation.cc:240] Negotiation successful
0806 11:38:13.686763 (+ 705us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4640,"thread_start_us":484,"threads_started":1}
I20260806 11:38:13.691872 1621 heartbeater.cc:347] Connected to a master server at 127.21.14.254:44873
I20260806 11:38:13.692303 1621 heartbeater.cc:464] Registering TS with master...
I20260806 11:38:13.693373 1621 heartbeater.cc:511] Master 127.21.14.254:44873 requested a full tablet report, sending...
I20260806 11:38:13.698045 1081 ts_manager.cc:194] Registered new tserver with Master: 1c01366def2a4081b18751497a17cb31 (127.21.14.195:45989)
I20260806 11:38:13.699038 1081 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.14.195@KRBTEST.COM'} at 127.21.14.195:35743
I20260806 11:38:13.709928 21563 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 11:38:13.734285 1625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:38:13.715008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38650 (local address 127.21.14.254:44873)
0806 11:38:13.715241 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0806 11:38:13.715246 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:38:13.716999 (+ 1753us) server_negotiation.cc:407] Connection header received
0806 11:38:13.717363 (+ 364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:38:13.717368 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:38:13.717464 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:38:13.717571 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:38:13.719331 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:38:13.720687 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:38:13.722717 (+ 2030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:38:13.723443 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:38:13.723607 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:38:13.727156 (+ 3549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:38:13.727162 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:38:13.727165 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:38:13.727222 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:38:13.730549 (+ 3327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:38:13.731610 (+ 1061us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:38:13.731614 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:38:13.731616 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:38:13.731809 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:38:13.732418 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:38:13.732422 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:38:13.732424 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:38:13.732884 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:38:13.733034 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 11:38:13.733531 (+ 497us) server_negotiation.cc:299] Negotiation successful
0806 11:38:13.733971 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260806 11:38:13.742254 1081 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38650:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 11:38:13.750164 1081 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 11:38:13.817250 1640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:38:13.783133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.194:37165 (local address 127.0.0.1:52110)
0806 11:38:13.797964 (+ 14831us) negotiation.cc:107] Waiting for socket to connect
0806 11:38:13.797993 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0806 11:38:13.798550 (+ 557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:38:13.802801 (+ 4251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:38:13.802804 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:38:13.802897 (+ 93us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:38:13.803069 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:38:13.803076 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:38:13.810733 (+ 7657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:38:13.810747 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:38:13.812175 (+ 1428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:38:13.812191 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:38:13.816588 (+ 4397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:38:13.816592 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:38:13.816743 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:38:13.816789 (+ 46us) client_negotiation.cc:769] Sending connection context
0806 11:38:13.816909 (+ 120us) client_negotiation.cc:240] Negotiation successful
0806 11:38:13.817072 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14654,"thread_start_us":743,"threads_started":1}
I20260806 11:38:13.819456 1637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:38:13.779507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.195:45989 (local address 127.0.0.1:39702)
0806 11:38:13.787556 (+ 8049us) negotiation.cc:107] Waiting for socket to connect
0806 11:38:13.787689 (+ 133us) client_negotiation.cc:174] Beginning negotiation
0806 11:38:13.788519 (+ 830us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:38:13.790920 (+ 2401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:38:13.790937 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:38:13.790986 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:38:13.791256 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:38:13.791265 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:38:13.812371 (+ 21106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:38:13.812386 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:38:13.813533 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:38:13.813547 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:38:13.818976 (+ 5429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:38:13.818981 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:38:13.819116 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:38:13.819144 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 11:38:13.819199 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 11:38:13.819280 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7574,"thread_start_us":210,"threads_started":1}
I20260806 11:38:13.820355 1638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:38:13.779859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39702 (local address 127.21.14.195:45989)
0806 11:38:13.788791 (+ 8932us) server_negotiation.cc:206] Beginning negotiation
0806 11:38:13.788908 (+ 117us) server_negotiation.cc:399] Waiting for connection header
0806 11:38:13.788955 (+ 47us) server_negotiation.cc:407] Connection header received
0806 11:38:13.789070 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:38:13.789075 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:38:13.789400 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:38:13.790192 (+ 792us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:38:13.795203 (+ 5011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:38:13.797385 (+ 2182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:38:13.813868 (+ 16483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:38:13.815083 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:38:13.819802 (+ 4719us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:38:13.819998 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0806 11:38:13.820066 (+ 68us) server_negotiation.cc:299] Negotiation successful
0806 11:38:13.820181 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8168,"thread_start_us":250,"threads_started":1}
I20260806 11:38:13.824273 1542 tablet_service.cc:1467] Processing CreateTablet for tablet 2de678d5477c49c68f3a471081ad5a28 (DEFAULT_TABLE table=test-table [id=cce2187c31d448a69774118671ac22f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:38:13.825675 1542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2de678d5477c49c68f3a471081ad5a28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:38:13.831787 1641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:38:13.784915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.193:41713 (local address 127.0.0.1:41852)
0806 11:38:13.800246 (+ 15331us) negotiation.cc:107] Waiting for socket to connect
0806 11:38:13.800280 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 11:38:13.800474 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:38:13.802260 (+ 1786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:38:13.802273 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:38:13.802304 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:38:13.802587 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:38:13.802605 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:38:13.807851 (+ 5246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:38:13.807865 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:38:13.810405 (+ 2540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:38:13.810435 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:38:13.831111 (+ 20676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:38:13.831117 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:38:13.831296 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:38:13.831357 (+ 61us) client_negotiation.cc:769] Sending connection context
0806 11:38:13.831499 (+ 142us) client_negotiation.cc:240] Negotiation successful
0806 11:38:13.831613 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15213,"thread_start_us":427,"threads_started":1}
I20260806 11:38:13.834362 1645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:38:13.790102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41852 (local address 127.21.14.193:41713)
0806 11:38:13.792173 (+ 2071us) server_negotiation.cc:206] Beginning negotiation
0806 11:38:13.792191 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 11:38:13.800694 (+ 8503us) server_negotiation.cc:407] Connection header received
0806 11:38:13.800907 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:38:13.800924 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:38:13.801329 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:38:13.801943 (+ 614us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:38:13.805600 (+ 3657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:38:13.807578 (+ 1978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:38:13.817683 (+ 10105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:38:13.818639 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:38:13.818761 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:38:13.833959 (+ 15198us) server_negotiation.cc:1035] Waiting for connection context
0806 11:38:13.834029 (+ 70us) server_negotiation.cc:299] Negotiation successful
0806 11:38:13.834176 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1839,"thread_start_us":394,"threads_started":1}
I20260806 11:38:13.838225 1245 tablet_service.cc:1467] Processing CreateTablet for tablet 2de678d5477c49c68f3a471081ad5a28 (DEFAULT_TABLE table=test-table [id=cce2187c31d448a69774118671ac22f1]), partition=RANGE (key) PARTITION UNBOUNDED
Aug 06 11:38:13 dist-test-slave-7c35 krb5kdc[572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016292, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.254@KRBTEST.COM for HTTP/127.21.14.212@KRBTEST.COM
I20260806 11:38:13.839306 1644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:38:13.790136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52110 (local address 127.21.14.194:37165)
0806 11:38:13.791757 (+ 1621us) server_negotiation.cc:206] Beginning negotiation
0806 11:38:13.791765 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 11:38:13.798738 (+ 6973us) server_negotiation.cc:407] Connection header received
0806 11:38:13.798964 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:38:13.798974 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:38:13.799466 (+ 492us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:38:13.799776 (+ 310us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:38:13.803307 (+ 3531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:38:13.805248 (+ 1941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:38:13.815239 (+ 9991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:38:13.816202 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:38:13.816322 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:38:13.816512 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0806 11:38:13.833310 (+ 16798us) server_negotiation.cc:299] Negotiation successful
0806 11:38:13.833489 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1414,"thread_start_us":303,"threads_started":1}
I20260806 11:38:13.839982 1245 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2de678d5477c49c68f3a471081ad5a28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:38:13.842972 1394 tablet_service.cc:1467] Processing CreateTablet for tablet 2de678d5477c49c68f3a471081ad5a28 (DEFAULT_TABLE table=test-table [id=cce2187c31d448a69774118671ac22f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:38:13.844344 1394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2de678d5477c49c68f3a471081ad5a28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:38:13.859150 1646 tablet_bootstrap.cc:492] T 2de678d5477c49c68f3a471081ad5a28 P 1c01366def2a4081b18751497a17cb31: Bootstrap starting.
I20260806 11:38:13.864081 1646 tablet_bootstrap.cc:654] T 2de678d5477c49c68f3a471081ad5a28 P 1c01366def2a4081b18751497a17cb31: Neither blocks nor log segments found. Creating new log.
I20260806 11:38:13.864759 1648 tablet_bootstrap.cc:492] T 2de678d5477c49c68f3a471081ad5a28 P 3af5d13ab5dd482a9b5926b8bdbcbbea: Bootstrap starting.
I20260806 11:38:13.865293 1646 log.cc:824] T 2de678d5477c49c68f3a471081ad5a28 P 1c01366def2a4081b18751497a17cb31: Log is configured to *not* fsync() on all Append() calls
I20260806 11:38:13.866323 1647 tablet_bootstrap.cc:492] T 2de678d5477c49c68f3a471081ad5a28 P d4b0211e0d17418e8360cc9ba778b3f3: Bootstrap starting.
I20260806 11:38:13.869673 1647 tablet_bootstrap.cc:654] T 2de678d5477c49c68f3a471081ad5a28 P d4b0211e0d17418e8360cc9ba778b3f3: Neither blocks nor log segments found. Creating new log.
I20260806 11:38:13.871099 1647 log.cc:824] T 2de678d5477c49c68f3a471081ad5a28 P d4b0211e0d17418e8360cc9ba778b3f3: Log is configured to *not* fsync() on all Append() calls
I20260806 11:38:13.875854 1648 tablet_bootstrap.cc:654] T 2de678d5477c49c68f3a471081ad5a28 P 3af5d13ab5dd482a9b5926b8bdbcbbea: Neither blocks nor log segments found. Creating new log.
I20260806 11:38:13.876978 1648 log.cc:824] T 2de678d5477c49c68f3a471081ad5a28 P 3af5d13ab5dd482a9b5926b8bdbcbbea: Log is configured to *not* fsync() on all Append() calls
I20260806 11:38:13.878079 1646 tablet_bootstrap.cc:492] T 2de678d5477c49c68f3a471081ad5a28 P 1c01366def2a4081b18751497a17cb31: No bootstrap required, opened a new log
I20260806 11:38:13.878433 1646 ts_tablet_manager.cc:1397] T 2de678d5477c49c68f3a471081ad5a28 P 1c01366def2a4081b18751497a17cb31: Time spent bootstrapping tablet: real 0.020s user 0.008s sys 0.000s
I20260806 11:38:13.880872 1647 tablet_bootstrap.cc:492] T 2de678d5477c49c68f3a471081ad5a28 P d4b0211e0d17418e8360cc9ba778b3f3: No bootstrap required, opened a new log
I20260806 11:38:13.881182 1647 ts_tablet_manager.cc:1397] T 2de678d5477c49c68f3a471081ad5a28 P d4b0211e0d17418e8360cc9ba778b3f3: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.000s
I20260806 11:38:13.884743 1646 raft_consensus.cc:348] T 2de678d5477c49c68f3a471081ad5a28 P 1c01366def2a4081b18751497a17cb31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c01366def2a4081b18751497a17cb31" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 45989 } } peers { permanent_uuid: "3af5d13ab5dd482a9b5926b8bdbcbbea" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37165 } } peers { permanent_uuid: "d4b0211e0d17418e8360cc9ba778b3f3" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 41713 } }
I20260806 11:38:13.885365 1646 raft_consensus.cc:374] T 2de678d5477c49c68f3a471081ad5a28 P 1c01366def2a4081b18751497a17cb31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:38:13.885509 1646 raft_consensus.cc:729] T 2de678d5477c49c68f3a471081ad5a28 P 1c01366def2a4081b18751497a17cb31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c01366def2a4081b18751497a17cb31, State: Initialized, Role: FOLLOWER
I20260806 11:38:13.886344 1646 consensus_queue.cc:260] T 2de678d5477c49c68f3a471081ad5a28 P 1c01366def2a4081b18751497a17cb31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c01366def2a4081b18751497a17cb31" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 45989 } } peers { permanent_uuid: "3af5d13ab5dd482a9b5926b8bdbcbbea" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37165 } } peers { permanent_uuid: "d4b0211e0d17418e8360cc9ba778b3f3" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 41713 } }
I20260806 11:38:13.888127 1646 ts_tablet_manager.cc:1428] T 2de678d5477c49c68f3a471081ad5a28 P 1c01366def2a4081b18751497a17cb31: Time spent starting tablet: real 0.010s user 0.002s sys 0.006s
I20260806 11:38:13.888656 1621 heartbeater.cc:503] Master 127.21.14.254:44873 was elected leader, sending a full tablet report...
I20260806 11:38:13.888715 1647 raft_consensus.cc:348] T 2de678d5477c49c68f3a471081ad5a28 P d4b0211e0d17418e8360cc9ba778b3f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c01366def2a4081b18751497a17cb31" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 45989 } } peers { permanent_uuid: "3af5d13ab5dd482a9b5926b8bdbcbbea" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37165 } } peers { permanent_uuid: "d4b0211e0d17418e8360cc9ba778b3f3" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 41713 } }
I20260806 11:38:13.889083 1647 raft_consensus.cc:374] T 2de678d5477c49c68f3a471081ad5a28 P d4b0211e0d17418e8360cc9ba778b3f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:38:13.889199 1647 raft_consensus.cc:729] T 2de678d5477c49c68f3a471081ad5a28 P d4b0211e0d17418e8360cc9ba778b3f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4b0211e0d17418e8360cc9ba778b3f3, State: Initialized, Role: FOLLOWER
I20260806 11:38:13.889850 1647 consensus_queue.cc:260] T 2de678d5477c49c68f3a471081ad5a28 P d4b0211e0d17418e8360cc9ba778b3f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c01366def2a4081b18751497a17cb31" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 45989 } } peers { permanent_uuid: "3af5d13ab5dd482a9b5926b8bdbcbbea" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37165 } } peers { permanent_uuid: "d4b0211e0d17418e8360cc9ba778b3f3" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 41713 } }
I20260806 11:38:13.891417 1647 ts_tablet_manager.cc:1428] T 2de678d5477c49c68f3a471081ad5a28 P d4b0211e0d17418e8360cc9ba778b3f3: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260806 11:38:13.898633 1648 tablet_bootstrap.cc:492] T 2de678d5477c49c68f3a471081ad5a28 P 3af5d13ab5dd482a9b5926b8bdbcbbea: No bootstrap required, opened a new log
I20260806 11:38:13.898947 1648 ts_tablet_manager.cc:1397] T 2de678d5477c49c68f3a471081ad5a28 P 3af5d13ab5dd482a9b5926b8bdbcbbea: Time spent bootstrapping tablet: real 0.035s user 0.008s sys 0.000s
I20260806 11:38:13.901379 1469 heartbeater.cc:503] Master 127.21.14.254:44873 was elected leader, sending a full tablet report...
W20260806 11:38:13.905171 1622 tablet.cc:2369] T 2de678d5477c49c68f3a471081ad5a28 P 1c01366def2a4081b18751497a17cb31: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 11:38:13.907217 1648 raft_consensus.cc:348] T 2de678d5477c49c68f3a471081ad5a28 P 3af5d13ab5dd482a9b5926b8bdbcbbea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c01366def2a4081b18751497a17cb31" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 45989 } } peers { permanent_uuid: "3af5d13ab5dd482a9b5926b8bdbcbbea" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37165 } } peers { permanent_uuid: "d4b0211e0d17418e8360cc9ba778b3f3" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 41713 } }
I20260806 11:38:13.907728 1648 raft_consensus.cc:374] T 2de678d5477c49c68f3a471081ad5a28 P 3af5d13ab5dd482a9b5926b8bdbcbbea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:38:13.907840 1648 raft_consensus.cc:729] T 2de678d5477c49c68f3a471081ad5a28 P 3af5d13ab5dd482a9b5926b8bdbcbbea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3af5d13ab5dd482a9b5926b8bdbcbbea, State: Initialized, Role: FOLLOWER
I20260806 11:38:13.908552 1648 consensus_queue.cc:260] T 2de678d5477c49c68f3a471081ad5a28 P 3af5d13ab5dd482a9b5926b8bdbcbbea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c01366def2a4081b18751497a17cb31" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 45989 } } peers { permanent_uuid: "3af5d13ab5dd482a9b5926b8bdbcbbea" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37165 } } peers { permanent_uuid: "d4b0211e0d17418e8360cc9ba778b3f3" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 41713 } }
I20260806 11:38:13.910190 1648 ts_tablet_manager.cc:1428] T 2de678d5477c49c68f3a471081ad5a28 P 3af5d13ab5dd482a9b5926b8bdbcbbea: Time spent starting tablet: real 0.011s user 0.002s sys 0.005s
W20260806 11:38:13.921383 1322 tablet.cc:2369] T 2de678d5477c49c68f3a471081ad5a28 P d4b0211e0d17418e8360cc9ba778b3f3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 11:38:13.971338 1653 raft_consensus.cc:482] T 2de678d5477c49c68f3a471081ad5a28 P d4b0211e0d17418e8360cc9ba778b3f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 11:38:13.971638 1653 raft_consensus.cc:504] T 2de678d5477c49c68f3a471081ad5a28 P d4b0211e0d17418e8360cc9ba778b3f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c01366def2a4081b18751497a17cb31" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 45989 } } peers { permanent_uuid: "3af5d13ab5dd482a9b5926b8bdbcbbea" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37165 } } peers { permanent_uuid: "d4b0211e0d17418e8360cc9ba778b3f3" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 41713 } }
I20260806 11:38:13.973038 1653 leader_election.cc:288] T 2de678d5477c49c68f3a471081ad5a28 P d4b0211e0d17418e8360cc9ba778b3f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1c01366def2a4081b18751497a17cb31 (127.21.14.195:45989), 3af5d13ab5dd482a9b5926b8bdbcbbea (127.21.14.194:37165)
I20260806 11:38:13.980211 1638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:38:13.974477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:54797 (local address 127.21.14.195:45989)
0806 11:38:13.974684 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0806 11:38:13.974692 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 11:38:13.975072 (+ 380us) server_negotiation.cc:407] Connection header received
0806 11:38:13.975275 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:38:13.975278 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:38:13.975329 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:38:13.975379 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:38:13.976031 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:38:13.976844 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:38:13.978756 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:38:13.979433 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:38:13.979539 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:38:13.979598 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 11:38:13.979968 (+ 370us) server_negotiation.cc:299] Negotiation successful
0806 11:38:13.980058 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 11:38:13.980940 1656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:38:13.973619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.195:45989 (local address 127.21.14.193:54797)
0806 11:38:13.974973 (+ 1354us) negotiation.cc:107] Waiting for socket to connect
0806 11:38:13.974987 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 11:38:13.975153 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:38:13.975551 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:38:13.975554 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:38:13.975574 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:38:13.975888 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:38:13.975897 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:38:13.977142 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:38:13.977146 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:38:13.978598 (+ 1452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:38:13.978611 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:38:13.979717 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:38:13.979720 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:38:13.979826 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:38:13.979850 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 11:38:13.980685 (+ 835us) client_negotiation.cc:240] Negotiation successful
0806 11:38:13.980776 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1281,"thread_start_us":125,"threads_started":1}
I20260806 11:38:13.982002 1549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2de678d5477c49c68f3a471081ad5a28" candidate_uuid: "d4b0211e0d17418e8360cc9ba778b3f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c01366def2a4081b18751497a17cb31" is_pre_election: true
I20260806 11:38:13.982436 1549 raft_consensus.cc:2454] T 2de678d5477c49c68f3a471081ad5a28 P 1c01366def2a4081b18751497a17cb31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4b0211e0d17418e8360cc9ba778b3f3 in term 0.
I20260806 11:38:13.983538 1190 leader_election.cc:302] T 2de678d5477c49c68f3a471081ad5a28 P d4b0211e0d17418e8360cc9ba778b3f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c01366def2a4081b18751497a17cb31, d4b0211e0d17418e8360cc9ba778b3f3; no voters:
I20260806 11:38:13.983891 1653 raft_consensus.cc:2781] T 2de678d5477c49c68f3a471081ad5a28 P d4b0211e0d17418e8360cc9ba778b3f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 11:38:13.983975 1653 raft_consensus.cc:482] T 2de678d5477c49c68f3a471081ad5a28 P d4b0211e0d17418e8360cc9ba778b3f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 11:38:13.984019 1653 raft_consensus.cc:3037] T 2de678d5477c49c68f3a471081ad5a28 P d4b0211e0d17418e8360cc9ba778b3f3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:38:13.985759 1653 raft_consensus.cc:504] T 2de678d5477c49c68f3a471081ad5a28 P d4b0211e0d17418e8360cc9ba778b3f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c01366def2a4081b18751497a17cb31" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 45989 } } peers { permanent_uuid: "3af5d13ab5dd482a9b5926b8bdbcbbea" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37165 } } peers { permanent_uuid: "d4b0211e0d17418e8360cc9ba778b3f3" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 41713 } }
I20260806 11:38:13.986162 1653 leader_election.cc:288] T 2de678d5477c49c68f3a471081ad5a28 P d4b0211e0d17418e8360cc9ba778b3f3 [CANDIDATE]: Term 1 election: Requested vote from peers 1c01366def2a4081b18751497a17cb31 (127.21.14.195:45989), 3af5d13ab5dd482a9b5926b8bdbcbbea (127.21.14.194:37165)
I20260806 11:38:13.987005 1549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2de678d5477c49c68f3a471081ad5a28" candidate_uuid: "d4b0211e0d17418e8360cc9ba778b3f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c01366def2a4081b18751497a17cb31"
I20260806 11:38:13.987160 1549 raft_consensus.cc:3037] T 2de678d5477c49c68f3a471081ad5a28 P 1c01366def2a4081b18751497a17cb31 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:38:13.989133 1549 raft_consensus.cc:2454] T 2de678d5477c49c68f3a471081ad5a28 P 1c01366def2a4081b18751497a17cb31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4b0211e0d17418e8360cc9ba778b3f3 in term 1.
I20260806 11:38:13.989674 1190 leader_election.cc:302] T 2de678d5477c49c68f3a471081ad5a28 P d4b0211e0d17418e8360cc9ba778b3f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c01366def2a4081b18751497a17cb31, d4b0211e0d17418e8360cc9ba778b3f3; no voters:
I20260806 11:38:13.989967 1653 raft_consensus.cc:2781] T 2de678d5477c49c68f3a471081ad5a28 P d4b0211e0d17418e8360cc9ba778b3f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:38:13.990412 1653 raft_consensus.cc:686] T 2de678d5477c49c68f3a471081ad5a28 P d4b0211e0d17418e8360cc9ba778b3f3 [term 1 LEADER]: Becoming Leader. State: Replica: d4b0211e0d17418e8360cc9ba778b3f3, State: Running, Role: LEADER
I20260806 11:38:13.990899 1653 consensus_queue.cc:237] T 2de678d5477c49c68f3a471081ad5a28 P d4b0211e0d17418e8360cc9ba778b3f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c01366def2a4081b18751497a17cb31" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 45989 } } peers { permanent_uuid: "3af5d13ab5dd482a9b5926b8bdbcbbea" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37165 } } peers { permanent_uuid: "d4b0211e0d17418e8360cc9ba778b3f3" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 41713 } }
I20260806 11:38:13.996043 1081 catalog_manager.cc:5495] T 2de678d5477c49c68f3a471081ad5a28 P d4b0211e0d17418e8360cc9ba778b3f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to d4b0211e0d17418e8360cc9ba778b3f3 (127.21.14.193). New cstate: current_term: 1 leader_uuid: "d4b0211e0d17418e8360cc9ba778b3f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c01366def2a4081b18751497a17cb31" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 45989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3af5d13ab5dd482a9b5926b8bdbcbbea" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37165 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4b0211e0d17418e8360cc9ba778b3f3" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 41713 } health_report { overall_health: HEALTHY } } }
I20260806 11:38:14.003582 1644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:38:13.974615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:38245 (local address 127.21.14.194:37165)
0806 11:38:13.974740 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 11:38:13.974745 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:38:13.981759 (+ 7014us) server_negotiation.cc:407] Connection header received
0806 11:38:13.981946 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:38:13.981950 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:38:13.982002 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:38:13.982048 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:38:13.982406 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:38:13.983249 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:38:14.001998 (+ 18749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:38:14.002697 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:38:14.002814 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:38:14.002907 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 11:38:14.003305 (+ 398us) server_negotiation.cc:299] Negotiation successful
0806 11:38:14.003405 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 11:38:14.004307 1656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:38:13.974016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.194:37165 (local address 127.21.14.193:38245)
0806 11:38:13.981679 (+ 7663us) negotiation.cc:107] Waiting for socket to connect
0806 11:38:13.981694 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 11:38:13.981834 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:38:13.982181 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:38:13.982183 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:38:13.982199 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:38:13.982276 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:38:13.982281 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:38:14.000257 (+ 17976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:38:14.000261 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:38:14.001780 (+ 1519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:38:14.001798 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:38:14.003033 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:38:14.003036 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:38:14.003137 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:38:14.003165 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 11:38:14.004048 (+ 883us) client_negotiation.cc:240] Negotiation successful
0806 11:38:14.004150 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7603,"thread_start_us":86,"threads_started":1}
I20260806 11:38:14.005333 1417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2de678d5477c49c68f3a471081ad5a28" candidate_uuid: "d4b0211e0d17418e8360cc9ba778b3f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3af5d13ab5dd482a9b5926b8bdbcbbea" is_pre_election: true
I20260806 11:38:14.005767 1417 raft_consensus.cc:2454] T 2de678d5477c49c68f3a471081ad5a28 P 3af5d13ab5dd482a9b5926b8bdbcbbea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4b0211e0d17418e8360cc9ba778b3f3 in term 0.
I20260806 11:38:14.006187 1404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2de678d5477c49c68f3a471081ad5a28" candidate_uuid: "d4b0211e0d17418e8360cc9ba778b3f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3af5d13ab5dd482a9b5926b8bdbcbbea"
I20260806 11:38:14.006392 1404 raft_consensus.cc:3037] T 2de678d5477c49c68f3a471081ad5a28 P 3af5d13ab5dd482a9b5926b8bdbcbbea [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:38:14.007954 1404 raft_consensus.cc:2454] T 2de678d5477c49c68f3a471081ad5a28 P 3af5d13ab5dd482a9b5926b8bdbcbbea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4b0211e0d17418e8360cc9ba778b3f3 in term 1.
I20260806 11:38:14.035126 1645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:38:14.028632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41854 (local address 127.21.14.193:41713)
0806 11:38:14.028767 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 11:38:14.028774 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 11:38:14.028791 (+ 17us) server_negotiation.cc:407] Connection header received
0806 11:38:14.029049 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:38:14.029053 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:38:14.029134 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:38:14.029253 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 11:38:14.029678 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:38:14.030566 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:38:14.032100 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:38:14.032446 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:38:14.032553 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:38:14.033040 (+ 487us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 11:38:14.033709 (+ 669us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 11:38:14.033828 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 11:38:14.034883 (+ 1055us) server_negotiation.cc:299] Negotiation successful
0806 11:38:14.034968 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 11:38:14.043468 1549 raft_consensus.cc:1261] T 2de678d5477c49c68f3a471081ad5a28 P 1c01366def2a4081b18751497a17cb31 [term 1 FOLLOWER]: Refusing update from remote peer d4b0211e0d17418e8360cc9ba778b3f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 11:38:14.043468 1404 raft_consensus.cc:1261] T 2de678d5477c49c68f3a471081ad5a28 P 3af5d13ab5dd482a9b5926b8bdbcbbea [term 1 FOLLOWER]: Refusing update from remote peer d4b0211e0d17418e8360cc9ba778b3f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 11:38:14.044523 1657 consensus_queue.cc:1035] T 2de678d5477c49c68f3a471081ad5a28 P d4b0211e0d17418e8360cc9ba778b3f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c01366def2a4081b18751497a17cb31" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 45989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 11:38:14.045178 1657 consensus_queue.cc:1035] T 2de678d5477c49c68f3a471081ad5a28 P d4b0211e0d17418e8360cc9ba778b3f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3af5d13ab5dd482a9b5926b8bdbcbbea" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 11:38:14.055830 1660 mvcc.cc:205] Tried to move back new op lower bound from 7315522740392697856 to 7315522740192665600. Current Snapshot: MvccSnapshot[applied={T|T < 7315522740392697856}]
I20260806 11:38:14.057443 1661 mvcc.cc:205] Tried to move back new op lower bound from 7315522740392697856 to 7315522740192665600. Current Snapshot: MvccSnapshot[applied={T|T < 7315522740392697856}]
I20260806 11:38:14.059811 1662 mvcc.cc:205] Tried to move back new op lower bound from 7315522740392697856 to 7315522740192665600. Current Snapshot: MvccSnapshot[applied={T|T < 7315522740392697856}]
I20260806 11:38:14.069209 1080 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38650:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 11:38:14.069422 1080 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38650:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 11:38:14.072852 1080 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9da2d9c342494b0da1108585a2c826d4: Sending DeleteTablet for 3 replicas of tablet 2de678d5477c49c68f3a471081ad5a28
I20260806 11:38:14.074236 1542 tablet_service.cc:1514] Processing DeleteTablet for tablet 2de678d5477c49c68f3a471081ad5a28 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:38:14 UTC) from {username='kudu', principal='kudu/127.21.14.254@KRBTEST.COM'} at 127.0.0.1:39702
I20260806 11:38:14.075094 1394 tablet_service.cc:1514] Processing DeleteTablet for tablet 2de678d5477c49c68f3a471081ad5a28 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:38:14 UTC) from {username='kudu', principal='kudu/127.21.14.254@KRBTEST.COM'} at 127.0.0.1:52110
I20260806 11:38:14.077021 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 1169
I20260806 11:38:14.080819 1670 tablet_replica.cc:330] T 2de678d5477c49c68f3a471081ad5a28 P 1c01366def2a4081b18751497a17cb31: stopping tablet replica
I20260806 11:38:14.081131 1671 tablet_replica.cc:330] T 2de678d5477c49c68f3a471081ad5a28 P 3af5d13ab5dd482a9b5926b8bdbcbbea: stopping tablet replica
I20260806 11:38:14.081593 1671 raft_consensus.cc:2229] T 2de678d5477c49c68f3a471081ad5a28 P 3af5d13ab5dd482a9b5926b8bdbcbbea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 11:38:14.082001 1670 raft_consensus.cc:2229] T 2de678d5477c49c68f3a471081ad5a28 P 1c01366def2a4081b18751497a17cb31 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 11:38:14.082623 1671 raft_consensus.cc:2258] T 2de678d5477c49c68f3a471081ad5a28 P 3af5d13ab5dd482a9b5926b8bdbcbbea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 11:38:14.083033 1670 raft_consensus.cc:2258] T 2de678d5477c49c68f3a471081ad5a28 P 1c01366def2a4081b18751497a17cb31 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 11:38:14.085330 1670 ts_tablet_manager.cc:1905] T 2de678d5477c49c68f3a471081ad5a28 P 1c01366def2a4081b18751497a17cb31: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 11:38:14.086879 1671 ts_tablet_manager.cc:1905] T 2de678d5477c49c68f3a471081ad5a28 P 3af5d13ab5dd482a9b5926b8bdbcbbea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 11:38:14.090461 1671 ts_tablet_manager.cc:1918] T 2de678d5477c49c68f3a471081ad5a28 P 3af5d13ab5dd482a9b5926b8bdbcbbea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260806 11:38:14.091535 1067 connection.cc:535] client connection to 127.21.14.193:41713 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 11:38:14.091727 1067 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 11:38:14.093132 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 1327
I20260806 11:38:14.095479 1670 ts_tablet_manager.cc:1918] T 2de678d5477c49c68f3a471081ad5a28 P 1c01366def2a4081b18751497a17cb31: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 11:38:14.095635 1670 log.cc:1196] T 2de678d5477c49c68f3a471081ad5a28 P 1c01366def2a4081b18751497a17cb31: Deleting WAL directory at /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786016169045816-21563-0/minicluster-data/ts-2/wal/wals/2de678d5477c49c68f3a471081ad5a28
I20260806 11:38:14.095963 1670 ts_tablet_manager.cc:1939] T 2de678d5477c49c68f3a471081ad5a28 P 1c01366def2a4081b18751497a17cb31: Deleting consensus metadata
I20260806 11:38:14.099464 1063 catalog_manager.cc:4841] TS 1c01366def2a4081b18751497a17cb31 (127.21.14.195:45989): tablet 2de678d5477c49c68f3a471081ad5a28 (table test-table [id=cce2187c31d448a69774118671ac22f1]) successfully deleted
I20260806 11:38:14.100288 1641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:38:14.099904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.193:41713 (local address 127.0.0.1:41864)
0806 11:38:14.100113 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0806 11:38:14.100195 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.193:41713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127}
W20260806 11:38:14.100656 1067 catalog_manager.cc:4584] TS d4b0211e0d17418e8360cc9ba778b3f3 (127.21.14.193:41713): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2de678d5477c49c68f3a471081ad5a28: Network error: Client connection negotiation failed: client connection to 127.21.14.193:41713: connect: Connection refused (error 111)
I20260806 11:38:14.106218 1641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:38:14.105882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.194:37165 (local address 127.0.0.1:52126)
0806 11:38:14.106040 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 11:38:14.106107 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.194:37165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260806 11:38:14.106366 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 1474
I20260806 11:38:14.116856 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 1046
I20260806 11:38:14.128226 21563 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 11:38:14.474526 21563 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 11:38:14.474747 21563 mini_ranger.cc:66] Stopping Ranger...
I20260806 11:38:14.831043 21563 mini_ranger.cc:68] Stopped Ranger
2026-08-06 11:38:14.831 UTC [604] LOG: received smart shutdown request
2026-08-06 11:38:14.836 UTC [604] LOG: background worker "logical replication launcher" (PID 613) exited with exit code 1
2026-08-06 11:38:14.837 UTC [608] LOG: shutting down
2026-08-06 11:38:14.981 UTC [604] LOG: database system is shut down
2026-08-06T11:38:14Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (57734 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 11:38:15 dist-test-slave-7c35 krb5kdc[1683](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 11:38:15 dist-test-slave-7c35 krb5kdc[1683](info): set up 2 sockets
Aug 06 11:38:15 dist-test-slave-7c35 krb5kdc[1683](info): commencing operation
krb5kdc: starting...
W20260806 11:38:16.217319 21563 mini_kdc.cc:121] Time spent starting KDC: real 1.089s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Aug 06 11:38:16 dist-test-slave-7c35 krb5kdc[1683](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016296, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T11:38:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T11:38:16Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.21.14.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.21.14.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.21.14.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/krb5kdc/rangeradmin_127.21.14.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.21.14.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/krb5kdc/rangeradmin_127.21.14.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.21.14.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.21.14.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.21.14.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/krb5kdc/rangerlookup_127.21.14.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.21.14.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/krb5kdc/rangerlookup_127.21.14.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.21.14.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.21.14.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.21.14.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/krb5kdc/HTTP_127.21.14.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.21.14.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/krb5kdc/HTTP_127.21.14.212@KRBTEST.COM.keytab.
I20260806 11:38:16.435359 21563 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task4_zrTV/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/postgres -l logfile start
2026-08-06 11:38:19.772 UTC [1717] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 11:38:19.772 UTC [1717] LOG: listening on IPv4 address "127.21.14.212", port 51829
2026-08-06 11:38:19.777 UTC [1717] LOG: listening on Unix socket "/tmp/.s.PGSQL.51829"
2026-08-06 11:38:19.802 UTC [1721] LOG: database system was shut down at 2026-08-06 11:38:17 UTC
2026-08-06 11:38:19.807 UTC [1717] LOG: database system is ready to accept connections
I20260806 11:38:20.797708 21563 mini_postgres.cc:96] Postgres bound to 51829
2026-08-06 11:38:20.805 UTC [1730] FATAL: database "slave" does not exist
127.21.14.212:51829 - accepting connections
I20260806 11:38:20.806500 21563 mini_ranger.cc:161] Starting Ranger...
I20260806 11:38:20.824445 21563 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 11:38:21.605597 21563 mini_ranger.cc:87] Created ranger Postgres database
I20260806 11:38:21.605679 21563 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ranger-admin
2026-08-06 11:38:21,918 [I] DB FLAVOR :POSTGRES
2026-08-06 11:38:21,918 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 11:38:21,919 [I] Checking connection..
2026-08-06 11:38:21,919 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4_zrTV/build/debug/bin/postgresql.jar:/tmp/dist-test-task4_zrTV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.14.212:51829/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 11:38:22,223 [I] Checking connection passed.
2026-08-06 11:38:22,224 [I] --------- Verifying version history table ---------
2026-08-06 11:38:22,224 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4_zrTV/build/debug/bin/postgresql.jar:/tmp/dist-test-task4_zrTV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.14.212:51829/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 11:38:22,503 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 11:38:22,504 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4_zrTV/build/debug/bin/postgresql.jar:/tmp/dist-test-task4_zrTV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.14.212:51829/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 11:38:22,776 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 11:38:22,776 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 11:38:22,776 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4_zrTV/build/debug/bin/postgresql.jar:/tmp/dist-test-task4_zrTV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.14.212:51829/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task4_zrTV/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 11:38:23.055 UTC [1803] WARNING: there is no transaction in progress
2026-08-06 11:38:23,063 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 11:38:23,064 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4_zrTV/build/debug/bin/postgresql.jar:/tmp/dist-test-task4_zrTV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.14.212:51829/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 11:38:23,376 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 11:38:23,377 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 11:38:23,377 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4_zrTV/build/debug/bin/postgresql.jar:/tmp/dist-test-task4_zrTV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.14.212:51829/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 11:38:23,656 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4_zrTV/build/debug/bin/postgresql.jar:/tmp/dist-test-task4_zrTV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.14.212:51829/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 11:38:23,939 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4_zrTV/build/debug/bin/postgresql.jar:/tmp/dist-test-task4_zrTV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.14.212:51829/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-7c35.c.gcp-upstream.internal','N') ;"
2026-08-06 11:38:24,223 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4_zrTV/build/debug/bin/postgresql.jar:/tmp/dist-test-task4_zrTV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.14.212:51829/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 11:38:24,508 [I] Table x_portal_user does not exist in database ranger
2026-08-06 11:38:24,509 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4_zrTV/build/debug/bin/postgresql.jar:/tmp/dist-test-task4_zrTV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.14.212:51829/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 11:38:24,784 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 11:38:24,784 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4_zrTV/build/debug/bin/postgresql.jar:/tmp/dist-test-task4_zrTV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.14.212:51829/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 11:38:25,052 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4_zrTV/build/debug/bin/postgresql.jar:/tmp/dist-test-task4_zrTV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.14.212:51829/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 11:38:25,323 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 11:38:25,323 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4_zrTV/build/debug/bin/postgresql.jar:/tmp/dist-test-task4_zrTV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.14.212:51829/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task4_zrTV/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 11:38:26.523 UTC [1922] WARNING: there is no transaction in progress
2026-08-06 11:38:26.544 UTC [1922] WARNING: there is no transaction in progress
2026-08-06 11:38:26.568 UTC [1922] WARNING: there is no transaction in progress
2026-08-06 11:38:26.587 UTC [1922] WARNING: there is no transaction in progress
2026-08-06 11:38:26.607 UTC [1922] WARNING: there is no transaction in progress
2026-08-06 11:38:26.631 UTC [1922] WARNING: there is no transaction in progress
2026-08-06 11:38:26.705 UTC [1922] WARNING: there is no transaction in progress
2026-08-06 11:38:26.714 UTC [1922] WARNING: there is no transaction in progress
2026-08-06 11:38:26.731 UTC [1922] WARNING: there is no transaction in progress
2026-08-06 11:38:26.742 UTC [1922] WARNING: there is no transaction in progress
2026-08-06 11:38:26.754 UTC [1922] WARNING: there is no transaction in progress
2026-08-06 11:38:26.773 UTC [1922] WARNING: there is no transaction in progress
2026-08-06 11:38:26.786 UTC [1922] WARNING: there is no transaction in progress
2026-08-06 11:38:26.798 UTC [1922] WARNING: there is no transaction in progress
2026-08-06 11:38:26.811 UTC [1922] WARNING: there is no transaction in progress
2026-08-06 11:38:26.822 UTC [1922] WARNING: there is no transaction in progress
2026-08-06 11:38:27,793 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 11:38:27,793 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4_zrTV/build/debug/bin/postgresql.jar:/tmp/dist-test-task4_zrTV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.14.212:51829/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 11:38:28,074 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 11:38:28,074 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4_zrTV/build/debug/bin/postgresql.jar:/tmp/dist-test-task4_zrTV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.14.212:51829/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 11:38:28,358 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 11:38:28,358 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4_zrTV/build/debug/bin/postgresql.jar:/tmp/dist-test-task4_zrTV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.14.212:51829/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 11:38:28,624 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 11:38:28,624 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4_zrTV/build/debug/bin/postgresql.jar:/tmp/dist-test-task4_zrTV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.14.212:51829/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 11:38:28,905 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4_zrTV/build/debug/bin/postgresql.jar:/tmp/dist-test-task4_zrTV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.14.212:51829/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 11:38:29,188 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4_zrTV/build/debug/bin/postgresql.jar:/tmp/dist-test-task4_zrTV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.14.212:51829/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-7c35.c.gcp-upstream.internal';"
2026-08-06 11:38:29,441 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 11:38:29,442 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4_zrTV/build/debug/bin/postgresql.jar:/tmp/dist-test-task4_zrTV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.14.212:51829/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 11:38:29,725 [I] DB_PATCHES have already been applied
I20260806 11:38:29.732731 21563 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ranger-admin:/tmp/dist-test-task4_zrTV/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task4_zrTV/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task4_zrTV/build/debug/bin/postgresql.jar:/tmp/dist-test-task4_zrTV/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 11:38:29.732847 21563 mini_ranger.cc:193] Using host: 127.21.14.212
I20260806 11:38:29.737947 21563 mini_ranger.cc:227] Ranger admin URL: http://127.21.14.212:38697
Aug 06, 2026 11:38:30 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 11:38:30 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 11:38:30 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 11:38:30 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 11:38:30 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 11:38:30 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 11:38:30 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 11:38:30.897229 21563 mini_ranger.cc:160] Time spent starting Ranger: real 10.091s user 0.000s sys 0.014s
Aug 06, 2026 11:38:52 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 11:38:52 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 06, 2026 11:38:52 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 11:38:52 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 11:38:52 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 11:38:52 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 11:38:52 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 11:38:52 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 11:38:52 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 11:38:52 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 11:38:52 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 11:38:52 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 11:38:52 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 11:38:52 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 11:38:52 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 11:38:52 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 11:38:52 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 11:38:53 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 11:38:54.216663 21563 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.21.14.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.21.14.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.21.14.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/krb5kdc/rangerkms_127.21.14.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.21.14.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/krb5kdc/rangerkms_127.21.14.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.21.14.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.21.14.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.21.14.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/krb5kdc/HTTP_127.21.14.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.21.14.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/krb5kdc/HTTP_127.21.14.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 11:38:54 dist-test-slave-7c35 krb5kdc[1683](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016334, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 11:38:54.333364 21563 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 11:38:54.346918 21563 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 11:38:55.571782 21563 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 11:38:55.571883 21563 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ranger-kms
I20260806 11:38:55.571908 21563 mini_ranger_kms.cc:227] Using postgres at 127.21.14.212:51829
2026-08-06 11:38:55,780 [I] DB FLAVOR :POSTGRES
2026-08-06 11:38:55,780 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 11:38:55,780 [I] Checking connection
2026-08-06 11:38:55,780 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4_zrTV/build/debug/bin/postgresql.jar:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.14.212:51829/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 11:38:56,052 [I] connection success
2026-08-06 11:38:56,052 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 11:38:56,052 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 11:38:56,052 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4_zrTV/build/debug/bin/postgresql.jar:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.14.212:51829/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 11:38:56,376 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 11:38:56,376 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 11:38:56,376 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 11:38:56,376 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4_zrTV/build/debug/bin/postgresql.jar:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.14.212:51829/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 11:38:56,684 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 11:38:57.306066 21563 mini_ranger_kms.cc:310] Created kms service
I20260806 11:38:57.538699 21563 mini_ranger_kms.cc:326] Created kudu user
I20260806 11:38:57.591966 21563 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 11:38:57 dist-test-slave-7c35 krb5kdc[1683](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016334, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.21.14.212@KRBTEST.COM
I20260806 11:38:58.039376 21563 mini_ranger_kms.cc:384] Added ranger policy
I20260806 11:38:58.039629 21563 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ranger-kms:/tmp/dist-test-task4_zrTV/build/debug/bin/postgresql.jar:/tmp/dist-test-task4_zrTV/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task4_zrTV/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 11:38:58.039652 21563 mini_ranger_kms.cc:240] Using host: 127.21.14.212
I20260806 11:38:58.044550 21563 mini_ranger_kms.cc:276] Ranger KMS PID: 2123
I20260806 11:38:58.044656 21563 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.21.14.212:49087
Aug 06, 2026 11:38:59 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 11:38:59 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 11:38:59 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 11:38:59 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 11:38:59 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 11:38:59 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260806 11:39:00.456805 21563 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.123s user 0.011s sys 0.180s
I20260806 11:39:00.456952 21563 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 11:39:00.457019 21563 mini_ranger_kms.cc:401] 127.21.14.212:49087/kms/v1/keys
Aug 06 11:39:00 dist-test-slave-7c35 krb5kdc[1683](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786016340, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.14.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 06 11:39:01 dist-test-slave-7c35 krb5kdc[1683](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016340, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.14.212@KRBTEST.COM for HTTP/127.21.14.212@KRBTEST.COM
Aug 06 11:39:01 dist-test-slave-7c35 krb5kdc[1683](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016340, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.14.212@KRBTEST.COM for HTTP/127.21.14.212@KRBTEST.COM
Aug 06 11:39:04 dist-test-slave-7c35 krb5kdc[1683](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016344, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.21.14.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab.
Entry for principal kudu/127.21.14.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab.
I20260806 11:39:04.067438 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:45429
--webserver_interface=127.21.14.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab
--principal=kudu/127.21.14.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:42779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.14.254:45429
--ranger_config_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.14.212:49087/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:39:04.200212 2178 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:39:04.200541 2178 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:39:04.200584 2178 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:39:04.205916 2178 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 11:39:04.206008 2178 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:39:04.206030 2178 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:39:04.206090 2178 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 11:39:04.206120 2178 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 11:39:04.212888 2178 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:42779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/master-0/wal
--ranger_kms_url=127.21.14.212:49087/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.21.14.254:45429
--ranger_config_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab
--principal=kudu/127.21.14.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:45429
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.14.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2178
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:39:04.214139 2178 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:39:04.215404 2178 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:39:04.226643 2186 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:39:04.226662 2183 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:39:04.226662 2184 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:39:04.226737 2178 server_base.cc:1034] running on GCE node
I20260806 11:39:04.227864 2178 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:39:04.229321 2178 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:39:04.230540 2178 hybrid_clock.cc:648] HybridClock initialized: now 1786016344230503 us; error 56 us; skew 500 ppm
Aug 06 11:39:04 dist-test-slave-7c35 krb5kdc[1683](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016344, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:39:04.240716 2178 init.cc:377] Logged in from keytab as kudu/127.21.14.254@KRBTEST.COM (short username kudu)
I20260806 11:39:04.241941 2178 webserver.cc:460] Webserver started at http://127.21.14.254:33673/ using document root <none> and password file <none>
I20260806 11:39:04.242604 2178 fs_manager.cc:359] Metadata directory not provided
I20260806 11:39:04.242697 2178 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:39:04.243037 2178 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 11:39:04 dist-test-slave-7c35 krb5kdc[1683](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016344, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.254@KRBTEST.COM for HTTP/127.21.14.212@KRBTEST.COM
I20260806 11:39:04.306643 2178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/master-0/data/instance:
uuid: "6cb55d7083d2414f8fd77a5beca4c46b"
format_stamp: "Formatted at 2026-08-06 11:39:04 on dist-test-slave-7c35"
server_key: "05fa88efd3af5192ffe8b805ac4acb8d"
server_key_iv: "02fc571fbc0c159daa88047d4fa27aae"
server_key_version: "kuduclusterkey@0"
I20260806 11:39:04.307490 2178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/master-0/wal/instance:
uuid: "6cb55d7083d2414f8fd77a5beca4c46b"
format_stamp: "Formatted at 2026-08-06 11:39:04 on dist-test-slave-7c35"
server_key: "05fa88efd3af5192ffe8b805ac4acb8d"
server_key_iv: "02fc571fbc0c159daa88047d4fa27aae"
server_key_version: "kuduclusterkey@0"
I20260806 11:39:04.312508 2178 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 11:39:04.329121 2193 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:39:04.331032 2178 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 11:39:04.331233 2178 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/master-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/master-0/wal
uuid: "6cb55d7083d2414f8fd77a5beca4c46b"
format_stamp: "Formatted at 2026-08-06 11:39:04 on dist-test-slave-7c35"
server_key: "05fa88efd3af5192ffe8b805ac4acb8d"
server_key_iv: "02fc571fbc0c159daa88047d4fa27aae"
server_key_version: "kuduclusterkey@0"
I20260806 11:39:04.331406 2178 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:39:04.366699 2178 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:39:04.370739 2178 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:39:04.371098 2178 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:39:04.380300 2178 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.254:45429
I20260806 11:39:04.380304 2250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.254:45429 every 8 connection(s)
I20260806 11:39:04.381485 2178 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
I20260806 11:39:04.384918 2251 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:39:04.388634 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 2178
I20260806 11:39:04.388805 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/master-0/wal/instance
I20260806 11:39:04.389137 21563 external_mini_cluster.cc:1428] Setting key 2fd0a2c5f9857bb8d5c2922f8660e1a7
I20260806 11:39:04.392052 2251 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6cb55d7083d2414f8fd77a5beca4c46b: Bootstrap starting.
I20260806 11:39:04.395524 2251 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6cb55d7083d2414f8fd77a5beca4c46b: Neither blocks nor log segments found. Creating new log.
I20260806 11:39:04.396605 2251 log.cc:824] T 00000000000000000000000000000000 P 6cb55d7083d2414f8fd77a5beca4c46b: Log is configured to *not* fsync() on all Append() calls
I20260806 11:39:04.399652 2251 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6cb55d7083d2414f8fd77a5beca4c46b: No bootstrap required, opened a new log
Aug 06 11:39:04 dist-test-slave-7c35 krb5kdc[1683](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016344, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:39:04.404238 2251 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6cb55d7083d2414f8fd77a5beca4c46b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cb55d7083d2414f8fd77a5beca4c46b" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 45429 } }
I20260806 11:39:04.404569 2251 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6cb55d7083d2414f8fd77a5beca4c46b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:39:04.404636 2251 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6cb55d7083d2414f8fd77a5beca4c46b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cb55d7083d2414f8fd77a5beca4c46b, State: Initialized, Role: FOLLOWER
I20260806 11:39:04.405191 2251 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6cb55d7083d2414f8fd77a5beca4c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cb55d7083d2414f8fd77a5beca4c46b" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 45429 } }
I20260806 11:39:04.405371 2251 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6cb55d7083d2414f8fd77a5beca4c46b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 11:39:04.405478 2251 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6cb55d7083d2414f8fd77a5beca4c46b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 11:39:04.405599 2251 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6cb55d7083d2414f8fd77a5beca4c46b [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:39:04.406838 2251 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6cb55d7083d2414f8fd77a5beca4c46b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cb55d7083d2414f8fd77a5beca4c46b" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 45429 } }
I20260806 11:39:04.407171 2254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:04.390747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38506 (local address 127.21.14.254:45429)
0806 11:39:04.391500 (+ 753us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:04.391531 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:04.391575 (+ 44us) server_negotiation.cc:407] Connection header received
0806 11:39:04.392666 (+ 1091us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:04.392692 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:04.393096 (+ 404us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:04.393427 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:39:04.394624 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:04.395852 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:04.397147 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:04.397898 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:04.398043 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:04.401634 (+ 3591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:39:04.401641 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:39:04.401659 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:39:04.401695 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:39:04.403744 (+ 2049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:04.404334 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:04.404341 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:04.404346 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:04.404471 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:04.404824 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:04.404828 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:04.404830 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:04.405297 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:39:04.405545 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:04.405896 (+ 351us) server_negotiation.cc:299] Negotiation successful
0806 11:39:04.406209 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":363,"threads_started":1}
I20260806 11:39:04.407332 2251 leader_election.cc:302] T 00000000000000000000000000000000 P 6cb55d7083d2414f8fd77a5beca4c46b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6cb55d7083d2414f8fd77a5beca4c46b; no voters:
I20260806 11:39:04.407776 2251 leader_election.cc:288] T 00000000000000000000000000000000 P 6cb55d7083d2414f8fd77a5beca4c46b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 11:39:04.407908 2256 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6cb55d7083d2414f8fd77a5beca4c46b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:39:04.408147 2256 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6cb55d7083d2414f8fd77a5beca4c46b [term 1 LEADER]: Becoming Leader. State: Replica: 6cb55d7083d2414f8fd77a5beca4c46b, State: Running, Role: LEADER
I20260806 11:39:04.408560 2256 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6cb55d7083d2414f8fd77a5beca4c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cb55d7083d2414f8fd77a5beca4c46b" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 45429 } }
I20260806 11:39:04.409281 2251 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6cb55d7083d2414f8fd77a5beca4c46b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 11:39:04.410326 2259 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6cb55d7083d2414f8fd77a5beca4c46b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6cb55d7083d2414f8fd77a5beca4c46b. Latest consensus state: current_term: 1 leader_uuid: "6cb55d7083d2414f8fd77a5beca4c46b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cb55d7083d2414f8fd77a5beca4c46b" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 45429 } } }
I20260806 11:39:04.410468 2259 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6cb55d7083d2414f8fd77a5beca4c46b [sys.catalog]: This master's current role is: LEADER
I20260806 11:39:04.410823 2258 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6cb55d7083d2414f8fd77a5beca4c46b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6cb55d7083d2414f8fd77a5beca4c46b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cb55d7083d2414f8fd77a5beca4c46b" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 45429 } } }
I20260806 11:39:04.410907 2267 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 11:39:04.410969 2258 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6cb55d7083d2414f8fd77a5beca4c46b [sys.catalog]: This master's current role is: LEADER
I20260806 11:39:04.414433 2267 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 11:39:04.414943 2251 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 11:39:04.429176 2267 catalog_manager.cc:1347] Generated new cluster ID: cf9fe3df754d441287d45251ccf765ef
I20260806 11:39:04.429272 2267 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 11:39:04.470476 2267 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 11:39:04.472213 2267 catalog_manager.cc:1504] Loading token signing keys...
I20260806 11:39:04.481762 2267 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6cb55d7083d2414f8fd77a5beca4c46b: Generated new TSK 0
I20260806 11:39:04.482784 2267 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.14.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab.
Entry for principal kudu/127.21.14.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab.
I20260806 11:39:05.204792 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.14.212:49087/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.193:0
--local_ip_for_outbound_sockets=127.21.14.193
--webserver_interface=127.21.14.193
--webserver_port=0
--tserver_master_addrs=127.21.14.254:45429
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab
--principal=kudu/127.21.14.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:42779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:39:05.388499 2300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:39:05.388988 2300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:39:05.389102 2300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:39:05.395990 2300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:39:05.396102 2300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:39:05.396202 2300 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.193
I20260806 11:39:05.402050 2300 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:42779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.21.14.212:49087/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab
--principal=kudu/127.21.14.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.14.193
--webserver_port=0
--tserver_master_addrs=127.21.14.254:45429
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2300
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.193
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:39:05.403434 2300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:39:05.404809 2300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:39:05.415971 2310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:39:05.415956 2307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:39:05.415956 2308 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:39:05.416769 2300 server_base.cc:1034] running on GCE node
I20260806 11:39:05.417367 2300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:39:05.418354 2300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:39:05.419592 2300 hybrid_clock.cc:648] HybridClock initialized: now 1786016345419547 us; error 54 us; skew 500 ppm
Aug 06 11:39:05 dist-test-slave-7c35 krb5kdc[1683](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016345, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:39:05.423578 2300 init.cc:377] Logged in from keytab as kudu/127.21.14.193@KRBTEST.COM (short username kudu)
I20260806 11:39:05.424661 2300 webserver.cc:460] Webserver started at http://127.21.14.193:41849/ using document root <none> and password file <none>
I20260806 11:39:05.425724 2300 fs_manager.cc:359] Metadata directory not provided
I20260806 11:39:05.425849 2300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:39:05.426146 2300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 11:39:05 dist-test-slave-7c35 krb5kdc[1683](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016345, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.193@KRBTEST.COM for HTTP/127.21.14.212@KRBTEST.COM
I20260806 11:39:05.443948 2300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-0/data/instance:
uuid: "8750dcbab26d4b049421d1db2624af86"
format_stamp: "Formatted at 2026-08-06 11:39:05 on dist-test-slave-7c35"
server_key: "fef8d42fd1c0af6bd80f43fec4c5f8a5"
server_key_iv: "e159949e4f304eb008c1c7e5b48fd364"
server_key_version: "kuduclusterkey@0"
I20260806 11:39:05.445032 2300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-0/wal/instance:
uuid: "8750dcbab26d4b049421d1db2624af86"
format_stamp: "Formatted at 2026-08-06 11:39:05 on dist-test-slave-7c35"
server_key: "fef8d42fd1c0af6bd80f43fec4c5f8a5"
server_key_iv: "e159949e4f304eb008c1c7e5b48fd364"
server_key_version: "kuduclusterkey@0"
I20260806 11:39:05.451861 2300 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260806 11:39:05.487638 2317 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:39:05.489791 2300 fs_manager.cc:725] Time spent opening block manager: real 0.015s user 0.004s sys 0.000s
I20260806 11:39:05.490000 2300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-0/wal
uuid: "8750dcbab26d4b049421d1db2624af86"
format_stamp: "Formatted at 2026-08-06 11:39:05 on dist-test-slave-7c35"
server_key: "fef8d42fd1c0af6bd80f43fec4c5f8a5"
server_key_iv: "e159949e4f304eb008c1c7e5b48fd364"
server_key_version: "kuduclusterkey@0"
I20260806 11:39:05.490190 2300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:39:05.528390 2300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:39:05.532793 2300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:39:05.533146 2300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:39:05.534217 2300 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:39:05.536023 2300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:39:05.536141 2300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:39:05.536274 2300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:39:05.536362 2300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:39:05.557225 2300 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.193:40305
I20260806 11:39:05.557374 2441 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.193:40305 every 8 connection(s)
I20260806 11:39:05.558714 2300 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
I20260806 11:39:05.565611 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 2300
I20260806 11:39:05.565743 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-0/wal/instance
I20260806 11:39:05.566049 21563 external_mini_cluster.cc:1428] Setting key d4d2fe05fbea8541f22569d4eeefd28f
Aug 06 11:39:05 dist-test-slave-7c35 krb5kdc[1683](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016345, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.193@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.14.194@KRBTEST.COM; defaulting to no policy
I20260806 11:39:05.582705 2446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:05.562619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:50515 (local address 127.21.14.254:45429)
0806 11:39:05.564686 (+ 2067us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:05.564691 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:05.564707 (+ 16us) server_negotiation.cc:407] Connection header received
0806 11:39:05.564757 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:05.564760 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:05.564815 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:05.564916 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:39:05.567300 (+ 2384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:05.568171 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:05.570454 (+ 2283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:05.570977 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:05.571095 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:05.575331 (+ 4236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:39:05.575337 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:39:05.575340 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:39:05.575375 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:39:05.577891 (+ 2516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:05.578762 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:05.578766 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:05.578769 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:05.578914 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:05.579458 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:05.579461 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:05.579463 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:05.579760 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:39:05.579889 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:05.582346 (+ 2457us) server_negotiation.cc:299] Negotiation successful
0806 11:39:05.582507 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1964,"thread_start_us":103,"threads_started":1}
I20260806 11:39:05.582898 2445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:05.561777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:45429 (local address 127.21.14.193:50515)
0806 11:39:05.562594 (+ 817us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:05.562661 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0806 11:39:05.564162 (+ 1501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:39:05.565203 (+ 1041us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:39:05.565213 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:39:05.565675 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:39:05.566634 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:05.567100 (+ 466us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:05.568427 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:05.568432 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:05.570170 (+ 1738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:05.570204 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:05.571301 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:05.571306 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:05.571561 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:05.572523 (+ 962us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:39:05.572546 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:39:05.575022 (+ 2476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:39:05.578078 (+ 3056us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:39:05.578086 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:39:05.578110 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:39:05.578614 (+ 504us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:39:05.579048 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:39:05.579052 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:39:05.579054 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:39:05.579308 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:39:05.579982 (+ 674us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:39:05.579990 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:39:05.580479 (+ 489us) client_negotiation.cc:769] Sending connection context
0806 11:39:05.580763 (+ 284us) client_negotiation.cc:240] Negotiation successful
0806 11:39:05.581236 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":202,"threads_started":1}
I20260806 11:39:05.584657 2442 heartbeater.cc:347] Connected to a master server at 127.21.14.254:45429
I20260806 11:39:05.584973 2442 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.194@KRBTEST.COM" created.
I20260806 11:39:05.585810 2442 heartbeater.cc:511] Master 127.21.14.254:45429 requested a full tablet report, sending...
I20260806 11:39:05.589830 2208 ts_manager.cc:194] Registered new tserver with Master: 8750dcbab26d4b049421d1db2624af86 (127.21.14.193:40305)
I20260806 11:39:05.591739 2208 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.14.193@KRBTEST.COM'} at 127.21.14.193:50515
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab.
Entry for principal kudu/127.21.14.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab.
I20260806 11:39:05.618016 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.14.212:49087/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.194:0
--local_ip_for_outbound_sockets=127.21.14.194
--webserver_interface=127.21.14.194
--webserver_port=0
--tserver_master_addrs=127.21.14.254:45429
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab
--principal=kudu/127.21.14.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:42779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 11:39:05 dist-test-slave-7c35 krb5kdc[1683](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786016345, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 11:39:05.790069 2449 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:39:05.790477 2449 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:39:05.790575 2449 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:39:05.795131 2449 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:39:05.795244 2449 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:39:05.795362 2449 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.194
I20260806 11:39:05.802917 2449 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:42779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.21.14.212:49087/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab
--principal=kudu/127.21.14.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.14.194
--webserver_port=0
--tserver_master_addrs=127.21.14.254:45429
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2449
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.194
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:39:05.804235 2449 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:39:05.805692 2449 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:39:05.819195 2455 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:39:05.820732 2449 server_base.cc:1034] running on GCE node
W20260806 11:39:05.822278 2459 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:39:05.822633 2456 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:39:05.823596 2449 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:39:05.824661 2449 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:39:05.825888 2449 hybrid_clock.cc:648] HybridClock initialized: now 1786016345825815 us; error 122 us; skew 500 ppm
Aug 06 11:39:05 dist-test-slave-7c35 krb5kdc[1683](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016345, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:39:05.829147 2449 init.cc:377] Logged in from keytab as kudu/127.21.14.194@KRBTEST.COM (short username kudu)
I20260806 11:39:05.830386 2449 webserver.cc:460] Webserver started at http://127.21.14.194:45447/ using document root <none> and password file <none>
I20260806 11:39:05.831108 2449 fs_manager.cc:359] Metadata directory not provided
I20260806 11:39:05.831207 2449 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:39:05.831476 2449 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 11:39:05 dist-test-slave-7c35 krb5kdc[1683](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016345, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.194@KRBTEST.COM for HTTP/127.21.14.212@KRBTEST.COM
I20260806 11:39:05.852371 2449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-1/data/instance:
uuid: "08f68b20f34146728d1be1ec93b4a6a2"
format_stamp: "Formatted at 2026-08-06 11:39:05 on dist-test-slave-7c35"
server_key: "cd6296cf8b31ae7731bb80122cb1b1d6"
server_key_iv: "e5cdb8de8fc8143b8043edff42d808e5"
server_key_version: "kuduclusterkey@0"
I20260806 11:39:05.853520 2449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-1/wal/instance:
uuid: "08f68b20f34146728d1be1ec93b4a6a2"
format_stamp: "Formatted at 2026-08-06 11:39:05 on dist-test-slave-7c35"
server_key: "cd6296cf8b31ae7731bb80122cb1b1d6"
server_key_iv: "e5cdb8de8fc8143b8043edff42d808e5"
server_key_version: "kuduclusterkey@0"
I20260806 11:39:05.859306 2449 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260806 11:39:05.883380 2466 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:39:05.885571 2449 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.003s sys 0.000s
I20260806 11:39:05.885838 2449 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-1/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-1/wal
uuid: "08f68b20f34146728d1be1ec93b4a6a2"
format_stamp: "Formatted at 2026-08-06 11:39:05 on dist-test-slave-7c35"
server_key: "cd6296cf8b31ae7731bb80122cb1b1d6"
server_key_iv: "e5cdb8de8fc8143b8043edff42d808e5"
server_key_version: "kuduclusterkey@0"
I20260806 11:39:05.886123 2449 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:39:05.913111 2449 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:39:05.917357 2449 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:39:05.917695 2449 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:39:05.918591 2449 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:39:05.919826 2449 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:39:05.919945 2449 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:39:05.920064 2449 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:39:05.920140 2449 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:39:05.932686 2449 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.194:34079
I20260806 11:39:05.934087 2449 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
I20260806 11:39:05.936467 2593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.194:34079 every 8 connection(s)
I20260806 11:39:05.944175 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 2449
I20260806 11:39:05.944311 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-1/wal/instance
I20260806 11:39:05.944624 21563 external_mini_cluster.cc:1428] Setting key e748bce5a11b845d1b91aa38069b9bfc
Aug 06 11:39:05 dist-test-slave-7c35 krb5kdc[1683](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016345, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.194@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:39:05.964222 2596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:05.940240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:45429 (local address 127.21.14.194:48183)
0806 11:39:05.942064 (+ 1824us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:05.942133 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0806 11:39:05.943392 (+ 1259us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:39:05.944587 (+ 1195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:39:05.944606 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:39:05.945453 (+ 847us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:39:05.949231 (+ 3778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:05.949267 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:05.950554 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:05.950557 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:05.952149 (+ 1592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:05.952165 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:05.953082 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:05.953087 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:05.953324 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:05.954135 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:39:05.954159 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:39:05.956089 (+ 1930us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:39:05.960276 (+ 4187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:39:05.960285 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:39:05.960309 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:39:05.960745 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:39:05.961139 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:39:05.961142 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:39:05.961144 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:39:05.961361 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:39:05.961906 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:39:05.961914 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:39:05.962311 (+ 397us) client_negotiation.cc:769] Sending connection context
0806 11:39:05.962569 (+ 258us) client_negotiation.cc:240] Negotiation successful
0806 11:39:05.963008 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1551,"thread_start_us":196,"threads_started":1}
I20260806 11:39:05.965221 2446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:05.941578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:48183 (local address 127.21.14.254:45429)
0806 11:39:05.941722 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:05.941728 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:05.942264 (+ 536us) server_negotiation.cc:407] Connection header received
0806 11:39:05.943761 (+ 1497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:05.943765 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:05.943836 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:05.943899 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:39:05.949485 (+ 5586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:05.950349 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:05.952331 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:05.952718 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:05.952812 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:05.956352 (+ 3540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:39:05.956357 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:39:05.956360 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:39:05.956395 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:39:05.960102 (+ 3707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:05.960895 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:05.960899 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:05.960901 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:05.961006 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:05.961478 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:05.961481 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:05.961483 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:05.961741 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:39:05.961842 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:05.964874 (+ 3032us) server_negotiation.cc:299] Negotiation successful
0806 11:39:05.965060 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 11:39:05.966509 2594 heartbeater.cc:347] Connected to a master server at 127.21.14.254:45429
I20260806 11:39:05.966799 2594 heartbeater.cc:464] Registering TS with master...
I20260806 11:39:05.967536 2594 heartbeater.cc:511] Master 127.21.14.254:45429 requested a full tablet report, sending...
I20260806 11:39:05.969009 2208 ts_manager.cc:194] Registered new tserver with Master: 08f68b20f34146728d1be1ec93b4a6a2 (127.21.14.194:34079)
I20260806 11:39:05.969766 2208 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.14.194@KRBTEST.COM'} at 127.21.14.194:48183
WARNING: no policy specified for kudu/127.21.14.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab.
Entry for principal kudu/127.21.14.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab.
I20260806 11:39:05.999725 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.14.212:49087/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.195:0
--local_ip_for_outbound_sockets=127.21.14.195
--webserver_interface=127.21.14.195
--webserver_port=0
--tserver_master_addrs=127.21.14.254:45429
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab
--principal=kudu/127.21.14.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:42779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:39:06.185536 2599 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:39:06.186081 2599 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:39:06.186203 2599 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:39:06.193506 2599 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:39:06.193713 2599 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:39:06.193848 2599 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.195
I20260806 11:39:06.203024 2599 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:42779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.21.14.212:49087/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab
--principal=kudu/127.21.14.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.14.195
--webserver_port=0
--tserver_master_addrs=127.21.14.254:45429
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2599
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.195
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:39:06.204943 2599 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:39:06.206564 2599 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:39:06.224166 2605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:39:06.224812 2604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:39:06.225607 2607 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:39:06.228873 2599 server_base.cc:1034] running on GCE node
I20260806 11:39:06.229777 2599 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:39:06.230765 2599 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:39:06.246284 2599 hybrid_clock.cc:648] HybridClock initialized: now 1786016346246136 us; error 147 us; skew 500 ppm
Aug 06 11:39:06 dist-test-slave-7c35 krb5kdc[1683](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016346, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:39:06.251049 2599 init.cc:377] Logged in from keytab as kudu/127.21.14.195@KRBTEST.COM (short username kudu)
I20260806 11:39:06.252451 2599 webserver.cc:460] Webserver started at http://127.21.14.195:39991/ using document root <none> and password file <none>
I20260806 11:39:06.253288 2599 fs_manager.cc:359] Metadata directory not provided
I20260806 11:39:06.253449 2599 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:39:06.253747 2599 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 11:39:06 dist-test-slave-7c35 krb5kdc[1683](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016346, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.195@KRBTEST.COM for HTTP/127.21.14.212@KRBTEST.COM
Aug 06 11:39:06 dist-test-slave-7c35 krb5kdc[1683](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016345, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.254@KRBTEST.COM for HTTP/127.21.14.212@KRBTEST.COM
I20260806 11:39:06.282048 2599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-2/data/instance:
uuid: "359a24fa76634a64ac889513d6a3158f"
format_stamp: "Formatted at 2026-08-06 11:39:06 on dist-test-slave-7c35"
server_key: "8d62e021a28a881a164dce0af90b2768"
server_key_iv: "72a51103fd9e660b4aa04f8b37c7e496"
server_key_version: "kuduclusterkey@0"
I20260806 11:39:06.297286 2599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-2/wal/instance:
uuid: "359a24fa76634a64ac889513d6a3158f"
format_stamp: "Formatted at 2026-08-06 11:39:06 on dist-test-slave-7c35"
server_key: "8d62e021a28a881a164dce0af90b2768"
server_key_iv: "72a51103fd9e660b4aa04f8b37c7e496"
server_key_version: "kuduclusterkey@0"
I20260806 11:39:06.314030 2599 fs_manager.cc:691] Time spent creating directory manager: real 0.013s user 0.004s sys 0.000s
I20260806 11:39:06.338661 2615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:39:06.340914 2599 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.002s sys 0.001s
I20260806 11:39:06.341101 2599 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-2/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-2/wal
uuid: "359a24fa76634a64ac889513d6a3158f"
format_stamp: "Formatted at 2026-08-06 11:39:06 on dist-test-slave-7c35"
server_key: "8d62e021a28a881a164dce0af90b2768"
server_key_iv: "72a51103fd9e660b4aa04f8b37c7e496"
server_key_version: "kuduclusterkey@0"
I20260806 11:39:06.341272 2599 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:39:06.359345 2599 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:39:06.364143 2599 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:39:06.364630 2599 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:39:06.365497 2599 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:39:06.366753 2599 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:39:06.366914 2599 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:39:06.367739 2599 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:39:06.367849 2599 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:39:06.381441 2599 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.195:32825
I20260806 11:39:06.383157 2599 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
I20260806 11:39:06.387205 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 2599
I20260806 11:39:06.387358 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-2/wal/instance
I20260806 11:39:06.387738 21563 external_mini_cluster.cc:1428] Setting key a748ca0b88a0a2303c67e420d3210d42
I20260806 11:39:06.389272 2739 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.195:32825 every 8 connection(s)
Aug 06 11:39:06 dist-test-slave-7c35 krb5kdc[1683](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016346, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.195@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:39:06.412288 2446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:06.393113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:40453 (local address 127.21.14.254:45429)
0806 11:39:06.393264 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:06.393269 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:06.394574 (+ 1305us) server_negotiation.cc:407] Connection header received
0806 11:39:06.395959 (+ 1385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:06.395963 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:06.396030 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:06.396107 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:39:06.397996 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:06.398894 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:06.402904 (+ 4010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:06.403279 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:06.403367 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:06.406727 (+ 3360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:39:06.406732 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:39:06.406735 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:39:06.406770 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:39:06.408824 (+ 2054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:06.409634 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:06.409638 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:06.409640 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:06.409740 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:06.410292 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:06.410296 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:06.410298 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:06.410561 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:39:06.410711 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:06.411924 (+ 1213us) server_negotiation.cc:299] Negotiation successful
0806 11:39:06.412101 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 11:39:06.414937 2742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:06.393782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:45429 (local address 127.21.14.195:40453)
0806 11:39:06.394382 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:06.394448 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0806 11:39:06.395649 (+ 1201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:39:06.396606 (+ 957us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:39:06.396618 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:39:06.397023 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:39:06.397782 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:06.397807 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:06.399113 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:06.399117 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:06.402718 (+ 3601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:06.402734 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:06.403625 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:06.403629 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:06.403873 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:06.404694 (+ 821us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:39:06.404718 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:39:06.406469 (+ 1751us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:39:06.409032 (+ 2563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:39:06.409040 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:39:06.409074 (+ 34us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:39:06.409487 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:39:06.409854 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:39:06.409858 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:39:06.409860 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:39:06.410167 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:39:06.410805 (+ 638us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:39:06.410814 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:39:06.411361 (+ 547us) client_negotiation.cc:769] Sending connection context
0806 11:39:06.413290 (+ 1929us) client_negotiation.cc:240] Negotiation successful
0806 11:39:06.413724 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":199,"threads_started":1}
I20260806 11:39:06.416553 2740 heartbeater.cc:347] Connected to a master server at 127.21.14.254:45429
I20260806 11:39:06.416828 2740 heartbeater.cc:464] Registering TS with master...
I20260806 11:39:06.417491 2740 heartbeater.cc:511] Master 127.21.14.254:45429 requested a full tablet report, sending...
I20260806 11:39:06.418840 2208 ts_manager.cc:194] Registered new tserver with Master: 359a24fa76634a64ac889513d6a3158f (127.21.14.195:32825)
I20260806 11:39:06.420365 2208 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.14.195@KRBTEST.COM'} at 127.21.14.195:40453
I20260806 11:39:06.426771 21563 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 11:39:06.427755 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 2300
I20260806 11:39:06.437088 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 2449
I20260806 11:39:06.444459 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 2599
I20260806 11:39:06.454500 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 2178
I20260806 11:39:06.465780 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:45429
--webserver_interface=127.21.14.254
--webserver_port=33673
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab
--principal=kudu/127.21.14.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:42779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.14.254:45429
--ranger_config_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.14.212:49087/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.21.14.212:49087/kms with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:39:06.588725 2744 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:39:06.589072 2744 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:39:06.589114 2744 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:39:06.593497 2744 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 11:39:06.593580 2744 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:39:06.593600 2744 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:39:06.593616 2744 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 11:39:06.593678 2744 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 11:39:06.599779 2744 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:42779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.21.14.212:49087/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.21.14.254:45429
--ranger_config_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab
--principal=kudu/127.21.14.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:45429
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.14.254
--webserver_port=33673
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2744
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:39:06.601060 2744 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:39:06.602218 2744 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:39:06.612181 2750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:39:06.612142 2752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:39:06.612152 2749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:39:06.612594 2744 server_base.cc:1034] running on GCE node
I20260806 11:39:06.613297 2744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:39:06.614689 2744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:39:06.615900 2744 hybrid_clock.cc:648] HybridClock initialized: now 1786016346615867 us; error 57 us; skew 500 ppm
Aug 06 11:39:06 dist-test-slave-7c35 krb5kdc[1683](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016346, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:39:06.619120 2744 init.cc:377] Logged in from keytab as kudu/127.21.14.254@KRBTEST.COM (short username kudu)
I20260806 11:39:06.620241 2744 webserver.cc:460] Webserver started at http://127.21.14.254:33673/ using document root <none> and password file <none>
I20260806 11:39:06.620854 2744 fs_manager.cc:359] Metadata directory not provided
I20260806 11:39:06.620961 2744 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 11:39:06 dist-test-slave-7c35 krb5kdc[1683](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016346, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.254@KRBTEST.COM for HTTP/127.21.14.212@KRBTEST.COM
I20260806 11:39:06.669654 2744 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 11:39:06.672338 2760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:39:06.673905 2744 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 11:39:06.674119 2744 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/master-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/master-0/wal
uuid: "6cb55d7083d2414f8fd77a5beca4c46b"
format_stamp: "Formatted at 2026-08-06 11:39:04 on dist-test-slave-7c35"
server_key: "05fa88efd3af5192ffe8b805ac4acb8d"
server_key_iv: "02fc571fbc0c159daa88047d4fa27aae"
server_key_version: "kuduclusterkey@0"
I20260806 11:39:06.674801 2744 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:39:06.721909 2744 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:39:06.725312 2744 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:39:06.725600 2744 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:39:06.733284 2744 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.254:45429
I20260806 11:39:06.733350 2822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.254:45429 every 8 connection(s)
I20260806 11:39:06.734474 2744 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
I20260806 11:39:06.736443 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 2744
I20260806 11:39:06.736937 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.14.212:49087/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.193:40305
--local_ip_for_outbound_sockets=127.21.14.193
--tserver_master_addrs=127.21.14.254:45429
--webserver_port=41849
--webserver_interface=127.21.14.193
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab
--principal=kudu/127.21.14.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:42779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.21.14.212:49087/kms with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 11:39:06.739388 2823 sys_catalog.cc:263] Verifying existing consensus state
I20260806 11:39:06.741231 2823 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6cb55d7083d2414f8fd77a5beca4c46b: Bootstrap starting.
I20260806 11:39:06.759054 2823 log.cc:824] T 00000000000000000000000000000000 P 6cb55d7083d2414f8fd77a5beca4c46b: Log is configured to *not* fsync() on all Append() calls
I20260806 11:39:06.766471 2823 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6cb55d7083d2414f8fd77a5beca4c46b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 11:39:06.767023 2823 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6cb55d7083d2414f8fd77a5beca4c46b: Bootstrap complete.
I20260806 11:39:06.771617 2823 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6cb55d7083d2414f8fd77a5beca4c46b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cb55d7083d2414f8fd77a5beca4c46b" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 45429 } }
I20260806 11:39:06.772074 2823 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6cb55d7083d2414f8fd77a5beca4c46b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cb55d7083d2414f8fd77a5beca4c46b, State: Initialized, Role: FOLLOWER
I20260806 11:39:06.772693 2823 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6cb55d7083d2414f8fd77a5beca4c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cb55d7083d2414f8fd77a5beca4c46b" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 45429 } }
I20260806 11:39:06.772861 2823 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6cb55d7083d2414f8fd77a5beca4c46b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 11:39:06.772948 2823 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6cb55d7083d2414f8fd77a5beca4c46b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 11:39:06.773078 2823 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6cb55d7083d2414f8fd77a5beca4c46b [term 1 FOLLOWER]: Advancing to term 2
I20260806 11:39:06.775265 2823 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6cb55d7083d2414f8fd77a5beca4c46b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cb55d7083d2414f8fd77a5beca4c46b" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 45429 } }
I20260806 11:39:06.775637 2823 leader_election.cc:302] T 00000000000000000000000000000000 P 6cb55d7083d2414f8fd77a5beca4c46b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6cb55d7083d2414f8fd77a5beca4c46b; no voters:
I20260806 11:39:06.775985 2823 leader_election.cc:288] T 00000000000000000000000000000000 P 6cb55d7083d2414f8fd77a5beca4c46b [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 11:39:06.776196 2827 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6cb55d7083d2414f8fd77a5beca4c46b [term 2 FOLLOWER]: Leader election won for term 2
I20260806 11:39:06.776464 2827 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6cb55d7083d2414f8fd77a5beca4c46b [term 2 LEADER]: Becoming Leader. State: Replica: 6cb55d7083d2414f8fd77a5beca4c46b, State: Running, Role: LEADER
I20260806 11:39:06.776839 2827 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6cb55d7083d2414f8fd77a5beca4c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cb55d7083d2414f8fd77a5beca4c46b" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 45429 } }
I20260806 11:39:06.777375 2823 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6cb55d7083d2414f8fd77a5beca4c46b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 11:39:06.778412 2828 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6cb55d7083d2414f8fd77a5beca4c46b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "6cb55d7083d2414f8fd77a5beca4c46b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cb55d7083d2414f8fd77a5beca4c46b" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 45429 } } }
I20260806 11:39:06.778569 2828 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6cb55d7083d2414f8fd77a5beca4c46b [sys.catalog]: This master's current role is: LEADER
I20260806 11:39:06.778980 2829 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6cb55d7083d2414f8fd77a5beca4c46b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6cb55d7083d2414f8fd77a5beca4c46b. Latest consensus state: current_term: 2 leader_uuid: "6cb55d7083d2414f8fd77a5beca4c46b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cb55d7083d2414f8fd77a5beca4c46b" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 45429 } } }
I20260806 11:39:06.779088 2829 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6cb55d7083d2414f8fd77a5beca4c46b [sys.catalog]: This master's current role is: LEADER
I20260806 11:39:06.779898 2837 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 11:39:06.780413 2823 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 11:39:06.787000 2837 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 11:39:06.788030 2837 catalog_manager.cc:1259] Loaded cluster ID: cf9fe3df754d441287d45251ccf765ef
I20260806 11:39:06.788122 2837 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 11:39:06.791615 2837 catalog_manager.cc:1504] Loading token signing keys...
I20260806 11:39:06.792568 2837 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6cb55d7083d2414f8fd77a5beca4c46b: Loaded TSK: 0
I20260806 11:39:06.793924 2837 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 11:39:06.872754 2825 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:39:06.873221 2825 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:39:06.873324 2825 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:39:06.878156 2825 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:39:06.878255 2825 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:39:06.878373 2825 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.193
I20260806 11:39:06.885671 2825 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:42779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.21.14.212:49087/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab
--principal=kudu/127.21.14.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.193:40305
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.14.193
--webserver_port=41849
--tserver_master_addrs=127.21.14.254:45429
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2825
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.193
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:39:06.887116 2825 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:39:06.888301 2825 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:39:06.901607 2864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:39:06.902268 2863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:39:06.902513 2866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:39:06.904036 2825 server_base.cc:1034] running on GCE node
I20260806 11:39:06.904654 2825 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:39:06.905474 2825 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:39:06.906689 2825 hybrid_clock.cc:648] HybridClock initialized: now 1786016346906645 us; error 54 us; skew 500 ppm
Aug 06 11:39:06 dist-test-slave-7c35 krb5kdc[1683](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016346, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:39:06.910665 2825 init.cc:377] Logged in from keytab as kudu/127.21.14.193@KRBTEST.COM (short username kudu)
I20260806 11:39:06.912014 2825 webserver.cc:460] Webserver started at http://127.21.14.193:41849/ using document root <none> and password file <none>
I20260806 11:39:06.912792 2825 fs_manager.cc:359] Metadata directory not provided
I20260806 11:39:06.912907 2825 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 11:39:06 dist-test-slave-7c35 krb5kdc[1683](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016346, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.193@KRBTEST.COM for HTTP/127.21.14.212@KRBTEST.COM
I20260806 11:39:06.944293 2825 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.000s
I20260806 11:39:06.948164 2876 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:39:06.950174 2825 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 11:39:06.951051 2825 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-0/wal
uuid: "8750dcbab26d4b049421d1db2624af86"
format_stamp: "Formatted at 2026-08-06 11:39:05 on dist-test-slave-7c35"
server_key: "fef8d42fd1c0af6bd80f43fec4c5f8a5"
server_key_iv: "e159949e4f304eb008c1c7e5b48fd364"
server_key_version: "kuduclusterkey@0"
I20260806 11:39:06.952025 2825 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:39:06.976158 2825 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:39:06.980294 2825 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:39:06.980557 2825 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:39:06.981315 2825 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:39:06.982364 2825 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:39:06.982429 2825 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:39:06.982506 2825 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:39:06.982533 2825 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:39:06.996676 2825 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.193:40305
I20260806 11:39:06.998193 2825 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
I20260806 11:39:06.999083 2995 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.193:40305 every 8 connection(s)
I20260806 11:39:07.006320 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 2825
I20260806 11:39:07.006919 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.14.212:49087/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.194:34079
--local_ip_for_outbound_sockets=127.21.14.194
--tserver_master_addrs=127.21.14.254:45429
--webserver_port=45447
--webserver_interface=127.21.14.194
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab
--principal=kudu/127.21.14.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:42779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.21.14.212:49087/kms with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 11:39:07 dist-test-slave-7c35 krb5kdc[1683](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016346, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.193@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:39:07.033609 2999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:07.004504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:48885 (local address 127.21.14.254:45429)
0806 11:39:07.005756 (+ 1252us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:07.005783 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:07.005890 (+ 107us) server_negotiation.cc:407] Connection header received
0806 11:39:07.007757 (+ 1867us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:07.007775 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:07.008323 (+ 548us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:07.008751 (+ 428us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:39:07.010888 (+ 2137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:07.012692 (+ 1804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:07.014784 (+ 2092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:07.015714 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:07.015925 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:07.022430 (+ 6505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:39:07.022442 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:39:07.022466 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:39:07.022517 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:39:07.025125 (+ 2608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:07.025995 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:07.026002 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:07.026007 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:07.026146 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:07.026588 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:07.026592 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:07.026594 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:07.027179 (+ 585us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:39:07.027315 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:07.032060 (+ 4745us) server_negotiation.cc:299] Negotiation successful
0806 11:39:07.032457 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1069,"thread_start_us":151,"threads_started":1}
I20260806 11:39:07.035527 2998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:07.002604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:45429 (local address 127.21.14.193:48885)
0806 11:39:07.005661 (+ 3057us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:07.005732 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0806 11:39:07.006978 (+ 1246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:39:07.009395 (+ 2417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:39:07.009406 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:39:07.009822 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:39:07.010582 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:07.010608 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:07.012929 (+ 2321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:07.012935 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:07.014569 (+ 1634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:07.014589 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:07.016227 (+ 1638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:07.016232 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:07.016456 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:07.017301 (+ 845us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:39:07.017326 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:39:07.022037 (+ 4711us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:39:07.025329 (+ 3292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:39:07.025338 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:39:07.025356 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:39:07.025809 (+ 453us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:39:07.026273 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:39:07.026277 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:39:07.026279 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:39:07.026452 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:39:07.031007 (+ 4555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:39:07.031017 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:39:07.031493 (+ 476us) client_negotiation.cc:769] Sending connection context
0806 11:39:07.034045 (+ 2552us) client_negotiation.cc:240] Negotiation successful
0806 11:39:07.034434 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2780,"thread_start_us":218,"threads_started":1}
I20260806 11:39:07.038486 2996 heartbeater.cc:347] Connected to a master server at 127.21.14.254:45429
I20260806 11:39:07.038779 2996 heartbeater.cc:464] Registering TS with master...
I20260806 11:39:07.039451 2996 heartbeater.cc:511] Master 127.21.14.254:45429 requested a full tablet report, sending...
W20260806 11:39:07.041951 2996 heartbeater.cc:644] Failed to heartbeat to 127.21.14.254:45429 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 11:39:07.042101 2996 heartbeater.cc:464] Registering TS with master...
I20260806 11:39:07.042313 2996 heartbeater.cc:511] Master 127.21.14.254:45429 requested a full tablet report, sending...
I20260806 11:39:07.043277 2996 heartbeater.cc:464] Registering TS with master...
I20260806 11:39:07.043474 2996 heartbeater.cc:511] Master 127.21.14.254:45429 requested a full tablet report, sending...
W20260806 11:39:07.044291 2996 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 11:39:07.191923 3000 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:39:07.192286 3000 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:39:07.192380 3000 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:39:07.197590 3000 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:39:07.197675 3000 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:39:07.197793 3000 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.194
I20260806 11:39:07.204715 3000 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:42779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.21.14.212:49087/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab
--principal=kudu/127.21.14.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.194:34079
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.14.194
--webserver_port=45447
--tserver_master_addrs=127.21.14.254:45429
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3000
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.194
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:39:07.206049 3000 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:39:07.207651 3000 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:39:07.219125 3006 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:39:07.219120 3005 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:39:07.219125 3008 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:39:07.219614 3000 server_base.cc:1034] running on GCE node
I20260806 11:39:07.220073 3000 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:39:07.220834 3000 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:39:07.222081 3000 hybrid_clock.cc:648] HybridClock initialized: now 1786016347222044 us; error 52 us; skew 500 ppm
Aug 06 11:39:07 dist-test-slave-7c35 krb5kdc[1683](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016347, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:39:07.225361 3000 init.cc:377] Logged in from keytab as kudu/127.21.14.194@KRBTEST.COM (short username kudu)
I20260806 11:39:07.226541 3000 webserver.cc:460] Webserver started at http://127.21.14.194:45447/ using document root <none> and password file <none>
I20260806 11:39:07.227260 3000 fs_manager.cc:359] Metadata directory not provided
I20260806 11:39:07.227391 3000 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 11:39:07 dist-test-slave-7c35 krb5kdc[1683](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016347, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.194@KRBTEST.COM for HTTP/127.21.14.212@KRBTEST.COM
I20260806 11:39:07.246038 3000 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 11:39:07.248976 3017 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:39:07.250638 3000 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 11:39:07.250819 3000 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-1/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-1/wal
uuid: "08f68b20f34146728d1be1ec93b4a6a2"
format_stamp: "Formatted at 2026-08-06 11:39:05 on dist-test-slave-7c35"
server_key: "cd6296cf8b31ae7731bb80122cb1b1d6"
server_key_iv: "e5cdb8de8fc8143b8043edff42d808e5"
server_key_version: "kuduclusterkey@0"
I20260806 11:39:07.251539 3000 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:39:07.266028 3000 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:39:07.270074 3000 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:39:07.270335 3000 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:39:07.271181 3000 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:39:07.272321 3000 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:39:07.272392 3000 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:39:07.272462 3000 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:39:07.272493 3000 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:39:07.286176 3000 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.194:34079
I20260806 11:39:07.286196 3138 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.194:34079 every 8 connection(s)
I20260806 11:39:07.287628 3000 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
I20260806 11:39:07.293852 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 3000
I20260806 11:39:07.294404 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.14.212:49087/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.195:32825
--local_ip_for_outbound_sockets=127.21.14.195
--tserver_master_addrs=127.21.14.254:45429
--webserver_port=39991
--webserver_interface=127.21.14.195
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab
--principal=kudu/127.21.14.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:42779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.21.14.212:49087/kms with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 11:39:07 dist-test-slave-7c35 krb5kdc[1683](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016347, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.194@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:39:07.309255 3141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:07.290839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:45429 (local address 127.21.14.194:40569)
0806 11:39:07.291489 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:07.291559 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0806 11:39:07.292533 (+ 974us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:39:07.293203 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:39:07.293211 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:39:07.293562 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:39:07.294467 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:07.295078 (+ 611us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:07.296359 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:07.296365 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:07.297551 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:07.297581 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:07.298689 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:07.298695 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:07.299015 (+ 320us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:07.299859 (+ 844us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:39:07.299888 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:39:07.301679 (+ 1791us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:39:07.304566 (+ 2887us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:39:07.304578 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:39:07.304608 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:39:07.305148 (+ 540us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:39:07.305592 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:39:07.305600 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:39:07.305604 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:39:07.305861 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:39:07.306531 (+ 670us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:39:07.306543 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:39:07.307074 (+ 531us) client_negotiation.cc:769] Sending connection context
0806 11:39:07.307351 (+ 277us) client_negotiation.cc:240] Negotiation successful
0806 11:39:07.307833 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":207,"threads_started":1}
I20260806 11:39:07.310339 2999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:07.290313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:40569 (local address 127.21.14.254:45429)
0806 11:39:07.290477 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:07.290482 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:07.291688 (+ 1206us) server_negotiation.cc:407] Connection header received
0806 11:39:07.292801 (+ 1113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:07.292804 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:07.292872 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:07.292980 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:39:07.295278 (+ 2298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:07.296146 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:07.297737 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:07.298175 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:07.298301 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:07.301983 (+ 3682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:39:07.301990 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:39:07.301996 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:39:07.302040 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:39:07.304333 (+ 2293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:07.305326 (+ 993us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:07.305333 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:07.305337 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:07.305451 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:07.306004 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:07.306010 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:07.306014 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:07.306341 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:39:07.306459 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:07.309850 (+ 3391us) server_negotiation.cc:299] Negotiation successful
0806 11:39:07.310057 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 11:39:07.312373 3139 heartbeater.cc:347] Connected to a master server at 127.21.14.254:45429
I20260806 11:39:07.312688 3139 heartbeater.cc:464] Registering TS with master...
I20260806 11:39:07.314687 3139 heartbeater.cc:511] Master 127.21.14.254:45429 requested a full tablet report, sending...
W20260806 11:39:07.317005 3139 heartbeater.cc:644] Failed to heartbeat to 127.21.14.254:45429 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 11:39:07.317167 3139 heartbeater.cc:464] Registering TS with master...
I20260806 11:39:07.317394 3139 heartbeater.cc:511] Master 127.21.14.254:45429 requested a full tablet report, sending...
I20260806 11:39:07.318315 3139 heartbeater.cc:464] Registering TS with master...
I20260806 11:39:07.318513 3139 heartbeater.cc:511] Master 127.21.14.254:45429 requested a full tablet report, sending...
W20260806 11:39:07.319311 3139 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 11:39:07.466023 3142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:39:07.466535 3142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:39:07.466619 3142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:39:07.473387 3142 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:39:07.473503 3142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:39:07.473601 3142 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.195
I20260806 11:39:07.482342 3142 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:42779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.21.14.212:49087/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab
--principal=kudu/127.21.14.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.195:32825
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.14.195
--webserver_port=39991
--tserver_master_addrs=127.21.14.254:45429
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3142
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.195
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:39:07.484247 3142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:39:07.485906 3142 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:39:07.499426 3148 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:39:07.500902 3142 server_base.cc:1034] running on GCE node
W20260806 11:39:07.499431 3152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:39:07.499444 3149 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:39:07.506826 3142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:39:07.507920 3142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:39:07.509146 3142 hybrid_clock.cc:648] HybridClock initialized: now 1786016347509105 us; error 61 us; skew 500 ppm
Aug 06 11:39:07 dist-test-slave-7c35 krb5kdc[1683](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016347, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:39:07.512897 3142 init.cc:377] Logged in from keytab as kudu/127.21.14.195@KRBTEST.COM (short username kudu)
I20260806 11:39:07.514084 3142 webserver.cc:460] Webserver started at http://127.21.14.195:39991/ using document root <none> and password file <none>
I20260806 11:39:07.514919 3142 fs_manager.cc:359] Metadata directory not provided
I20260806 11:39:07.515102 3142 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 11:39:07 dist-test-slave-7c35 krb5kdc[1683](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016347, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.195@KRBTEST.COM for HTTP/127.21.14.212@KRBTEST.COM
I20260806 11:39:07.540333 3142 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.004s sys 0.000s
I20260806 11:39:07.544004 3160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:39:07.545735 3142 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 11:39:07.545922 3142 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-2/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-2/wal
uuid: "359a24fa76634a64ac889513d6a3158f"
format_stamp: "Formatted at 2026-08-06 11:39:06 on dist-test-slave-7c35"
server_key: "8d62e021a28a881a164dce0af90b2768"
server_key_iv: "72a51103fd9e660b4aa04f8b37c7e496"
server_key_version: "kuduclusterkey@0"
I20260806 11:39:07.546602 3142 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:39:07.561882 3142 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:39:07.565637 3142 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:39:07.565872 3142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:39:07.566838 3142 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:39:07.568434 3142 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:39:07.568516 3142 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:39:07.568580 3142 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:39:07.568615 3142 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:39:07.581439 3142 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.195:32825
I20260806 11:39:07.581465 3279 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.195:32825 every 8 connection(s)
I20260806 11:39:07.582607 3142 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
I20260806 11:39:07.590655 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 3142
Aug 06 11:39:07 dist-test-slave-7c35 krb5kdc[1683](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016347, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.195@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:39:07.603286 2999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:07.585779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:43865 (local address 127.21.14.254:45429)
0806 11:39:07.585970 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:07.585976 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:07.586956 (+ 980us) server_negotiation.cc:407] Connection header received
0806 11:39:07.588020 (+ 1064us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:07.588026 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:07.588104 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:07.588241 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:39:07.589534 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:07.590433 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:07.592944 (+ 2511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:07.593328 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:07.593484 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:07.596739 (+ 3255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:39:07.596744 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:39:07.596747 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:39:07.596779 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:39:07.598687 (+ 1908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:07.600946 (+ 2259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:07.600951 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:07.600952 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:07.601058 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:07.601595 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:07.601599 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:07.601601 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:07.601867 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:39:07.601966 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:07.602916 (+ 950us) server_negotiation.cc:299] Negotiation successful
0806 11:39:07.603082 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 11:39:07.605055 3282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:07.586196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:45429 (local address 127.21.14.195:43865)
0806 11:39:07.586755 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:07.586813 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 11:39:07.587770 (+ 957us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:39:07.588487 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:39:07.588495 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:39:07.588791 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:39:07.589339 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:07.589354 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:07.590630 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:07.590634 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:07.592346 (+ 1712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:07.592365 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:07.593536 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:07.593542 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:07.593790 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:07.594596 (+ 806us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:39:07.594620 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:39:07.596489 (+ 1869us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:39:07.600252 (+ 3763us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:39:07.600261 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:39:07.600287 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:39:07.600781 (+ 494us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:39:07.601190 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:39:07.601195 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:39:07.601198 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:39:07.601399 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:39:07.602042 (+ 643us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:39:07.602048 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:39:07.602503 (+ 455us) client_negotiation.cc:769] Sending connection context
0806 11:39:07.603617 (+ 1114us) client_negotiation.cc:240] Negotiation successful
0806 11:39:07.603997 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":182,"threads_started":1}
I20260806 11:39:07.606665 3280 heartbeater.cc:347] Connected to a master server at 127.21.14.254:45429
I20260806 11:39:07.607002 3280 heartbeater.cc:464] Registering TS with master...
I20260806 11:39:07.607713 3280 heartbeater.cc:511] Master 127.21.14.254:45429 requested a full tablet report, sending...
W20260806 11:39:07.609704 3280 heartbeater.cc:644] Failed to heartbeat to 127.21.14.254:45429 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 11:39:07.609869 3280 heartbeater.cc:464] Registering TS with master...
I20260806 11:39:07.610133 3280 heartbeater.cc:511] Master 127.21.14.254:45429 requested a full tablet report, sending...
I20260806 11:39:07.611138 3280 heartbeater.cc:464] Registering TS with master...
I20260806 11:39:07.611374 3280 heartbeater.cc:511] Master 127.21.14.254:45429 requested a full tablet report, sending...
W20260806 11:39:07.612291 3280 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 11:39:07.619406 3283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:07.591491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38522 (local address 127.21.14.254:45429)
0806 11:39:07.592667 (+ 1176us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:07.592671 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:07.593116 (+ 445us) server_negotiation.cc:407] Connection header received
0806 11:39:07.593164 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:07.593167 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:07.593222 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:07.593318 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:39:07.594421 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:07.596771 (+ 2350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:07.600946 (+ 4175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:07.601348 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:07.601480 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:07.603755 (+ 2275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:39:07.603759 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:39:07.603763 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:39:07.603795 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:39:07.605977 (+ 2182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:07.613375 (+ 7398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:07.613379 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:07.613381 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:07.613587 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:07.617217 (+ 3630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:07.617224 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:07.617225 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:07.617646 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:39:07.617920 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:07.618501 (+ 581us) server_negotiation.cc:299] Negotiation successful
0806 11:39:07.618775 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1077,"thread_start_us":122,"threads_started":1}
I20260806 11:39:08.044550 2996 heartbeater.cc:464] Registering TS with master...
I20260806 11:39:08.044947 2996 heartbeater.cc:511] Master 127.21.14.254:45429 requested a full tablet report, sending...
I20260806 11:39:08.046469 2783 ts_manager.cc:194] Registered new tserver with Master: 8750dcbab26d4b049421d1db2624af86 (127.21.14.193:40305)
I20260806 11:39:08.048477 2783 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.14.193@KRBTEST.COM'} at 127.21.14.193:48885
Aug 06 11:39:08 dist-test-slave-7c35 krb5kdc[1683](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786016348, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:39:08.319556 3139 heartbeater.cc:464] Registering TS with master...
I20260806 11:39:08.319943 3139 heartbeater.cc:511] Master 127.21.14.254:45429 requested a full tablet report, sending...
I20260806 11:39:08.320830 2783 ts_manager.cc:194] Registered new tserver with Master: 08f68b20f34146728d1be1ec93b4a6a2 (127.21.14.194:34079)
I20260806 11:39:08.321638 2783 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.14.194@KRBTEST.COM'} at 127.21.14.194:40569
Aug 06 11:39:08 dist-test-slave-7c35 krb5kdc[1683](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016348, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.254@KRBTEST.COM for HTTP/127.21.14.212@KRBTEST.COM
I20260806 11:39:08.612576 3280 heartbeater.cc:464] Registering TS with master...
I20260806 11:39:08.612931 3280 heartbeater.cc:511] Master 127.21.14.254:45429 requested a full tablet report, sending...
I20260806 11:39:08.614063 2783 ts_manager.cc:194] Registered new tserver with Master: 359a24fa76634a64ac889513d6a3158f (127.21.14.195:32825)
I20260806 11:39:08.615021 2783 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.14.195@KRBTEST.COM'} at 127.21.14.195:43865
I20260806 11:39:08.622601 21563 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 11:39:08.636489 3303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:08.624769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51066 (local address 127.21.14.254:45429)
0806 11:39:08.625079 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:08.625084 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:08.625134 (+ 50us) server_negotiation.cc:407] Connection header received
0806 11:39:08.625300 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:08.625304 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:08.625369 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:08.625491 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:39:08.626351 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:08.627443 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:08.628806 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:08.629562 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:08.629705 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:08.631851 (+ 2146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:39:08.631855 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:39:08.631858 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:39:08.631896 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:39:08.634068 (+ 2172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:08.634716 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:08.634720 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:08.634722 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:08.634836 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:08.635210 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:08.635213 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:08.635215 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:08.635474 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:39:08.635630 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:08.635952 (+ 322us) server_negotiation.cc:299] Negotiation successful
0806 11:39:08.636104 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":102,"threads_started":1}
I20260806 11:39:08.641465 2783 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51066:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 11:39:08.646773 2783 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 11:39:08.670912 3311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:08.661390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32830 (local address 127.21.14.194:34079)
0806 11:39:08.662111 (+ 721us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:08.662118 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:08.663151 (+ 1033us) server_negotiation.cc:407] Connection header received
0806 11:39:08.663461 (+ 310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:08.663467 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:08.663706 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:08.663836 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:39:08.665165 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:08.666272 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:08.668909 (+ 2637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:08.669819 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:08.669910 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:08.670077 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:08.670586 (+ 509us) server_negotiation.cc:299] Negotiation successful
0806 11:39:08.670711 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":617,"thread_start_us":152,"threads_started":1}
I20260806 11:39:08.672456 3308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:08.661086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.194:34079 (local address 127.0.0.1:32830)
0806 11:39:08.663037 (+ 1951us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:08.663052 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 11:39:08.663274 (+ 222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:39:08.664647 (+ 1373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:39:08.664650 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:39:08.664669 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:39:08.664764 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:08.664769 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:08.666432 (+ 1663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:08.666435 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:08.668741 (+ 2306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:08.668755 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:08.670209 (+ 1454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:08.670213 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:08.670339 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:08.670378 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 11:39:08.671113 (+ 735us) client_negotiation.cc:240] Negotiation successful
0806 11:39:08.672301 (+ 1188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1881,"thread_start_us":83,"threads_started":1}
I20260806 11:39:08.675612 3310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:08.661103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51026 (local address 127.21.14.193:40305)
0806 11:39:08.661903 (+ 800us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:08.661911 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:08.663163 (+ 1252us) server_negotiation.cc:407] Connection header received
0806 11:39:08.663699 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:08.663706 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:08.663916 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:08.664049 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:39:08.664663 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:08.665869 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:08.672531 (+ 6662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:08.673468 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:08.673581 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:08.673754 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:08.675281 (+ 1527us) server_negotiation.cc:299] Negotiation successful
0806 11:39:08.675427 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":678,"thread_start_us":145,"threads_started":1}
I20260806 11:39:08.676271 3309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:08.660971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.193:40305 (local address 127.0.0.1:51026)
0806 11:39:08.662983 (+ 2012us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:08.663047 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0806 11:39:08.663505 (+ 458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:39:08.664316 (+ 811us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:39:08.664322 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:39:08.664350 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:39:08.664518 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:08.664528 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:08.666029 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:08.666033 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:08.667812 (+ 1779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:08.667825 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:08.674951 (+ 7126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:08.674955 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:08.675069 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:08.675099 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 11:39:08.675998 (+ 899us) client_negotiation.cc:240] Negotiation successful
0806 11:39:08.676109 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1783,"mutex_wait_us":201,"thread_start_us":140,"threads_started":1}
I20260806 11:39:08.678606 3313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:08.662393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37160 (local address 127.21.14.195:32825)
0806 11:39:08.662775 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:08.662782 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:08.665079 (+ 2297us) server_negotiation.cc:407] Connection header received
0806 11:39:08.668044 (+ 2965us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:08.668052 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:08.668320 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:08.668630 (+ 310us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:39:08.669215 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:08.670441 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:08.676690 (+ 6249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:08.677637 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:08.677747 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:08.677905 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:08.678335 (+ 430us) server_negotiation.cc:299] Negotiation successful
0806 11:39:08.678448 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":144,"threads_started":1}
I20260806 11:39:08.679261 3312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:08.662167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.195:32825 (local address 127.0.0.1:37160)
0806 11:39:08.664976 (+ 2809us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:08.664990 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 11:39:08.666642 (+ 1652us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:39:08.668916 (+ 2274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:39:08.668919 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:39:08.668942 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:39:08.669061 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:08.669069 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:08.670613 (+ 1544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:08.670616 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:08.672328 (+ 1712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:08.672342 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:08.678033 (+ 5691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:08.678036 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:08.678151 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:08.678176 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 11:39:08.679016 (+ 840us) client_negotiation.cc:240] Negotiation successful
0806 11:39:08.679116 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2746,"thread_start_us":97,"threads_started":1}
I20260806 11:39:08.677734 3068 tablet_service.cc:1467] Processing CreateTablet for tablet 08ed7c66e9024375b7965c1f7f5dbd4a (DEFAULT_TABLE table=test-table [id=e07d03ce03ad4220bd7d2495a88b0332]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:39:08.680356 3068 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08ed7c66e9024375b7965c1f7f5dbd4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:39:08.680465 2927 tablet_service.cc:1467] Processing CreateTablet for tablet 08ed7c66e9024375b7965c1f7f5dbd4a (DEFAULT_TABLE table=test-table [id=e07d03ce03ad4220bd7d2495a88b0332]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:39:08.681740 2927 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08ed7c66e9024375b7965c1f7f5dbd4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:39:08.682957 3212 tablet_service.cc:1467] Processing CreateTablet for tablet 08ed7c66e9024375b7965c1f7f5dbd4a (DEFAULT_TABLE table=test-table [id=e07d03ce03ad4220bd7d2495a88b0332]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:39:08.685427 3212 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08ed7c66e9024375b7965c1f7f5dbd4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:39:08.702322 3314 tablet_bootstrap.cc:492] T 08ed7c66e9024375b7965c1f7f5dbd4a P 8750dcbab26d4b049421d1db2624af86: Bootstrap starting.
I20260806 11:39:08.704826 3318 tablet_bootstrap.cc:492] T 08ed7c66e9024375b7965c1f7f5dbd4a P 08f68b20f34146728d1be1ec93b4a6a2: Bootstrap starting.
I20260806 11:39:08.705549 3314 tablet_bootstrap.cc:654] T 08ed7c66e9024375b7965c1f7f5dbd4a P 8750dcbab26d4b049421d1db2624af86: Neither blocks nor log segments found. Creating new log.
I20260806 11:39:08.706610 3314 log.cc:824] T 08ed7c66e9024375b7965c1f7f5dbd4a P 8750dcbab26d4b049421d1db2624af86: Log is configured to *not* fsync() on all Append() calls
I20260806 11:39:08.707436 3318 tablet_bootstrap.cc:654] T 08ed7c66e9024375b7965c1f7f5dbd4a P 08f68b20f34146728d1be1ec93b4a6a2: Neither blocks nor log segments found. Creating new log.
I20260806 11:39:08.707546 3317 tablet_bootstrap.cc:492] T 08ed7c66e9024375b7965c1f7f5dbd4a P 359a24fa76634a64ac889513d6a3158f: Bootstrap starting.
I20260806 11:39:08.708640 3318 log.cc:824] T 08ed7c66e9024375b7965c1f7f5dbd4a P 08f68b20f34146728d1be1ec93b4a6a2: Log is configured to *not* fsync() on all Append() calls
I20260806 11:39:08.712015 3314 tablet_bootstrap.cc:492] T 08ed7c66e9024375b7965c1f7f5dbd4a P 8750dcbab26d4b049421d1db2624af86: No bootstrap required, opened a new log
I20260806 11:39:08.712263 3318 tablet_bootstrap.cc:492] T 08ed7c66e9024375b7965c1f7f5dbd4a P 08f68b20f34146728d1be1ec93b4a6a2: No bootstrap required, opened a new log
I20260806 11:39:08.712275 3314 ts_tablet_manager.cc:1397] T 08ed7c66e9024375b7965c1f7f5dbd4a P 8750dcbab26d4b049421d1db2624af86: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260806 11:39:08.712473 3318 ts_tablet_manager.cc:1397] T 08ed7c66e9024375b7965c1f7f5dbd4a P 08f68b20f34146728d1be1ec93b4a6a2: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260806 11:39:08.717024 3314 raft_consensus.cc:348] T 08ed7c66e9024375b7965c1f7f5dbd4a P 8750dcbab26d4b049421d1db2624af86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8750dcbab26d4b049421d1db2624af86" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40305 } } peers { permanent_uuid: "08f68b20f34146728d1be1ec93b4a6a2" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 34079 } } peers { permanent_uuid: "359a24fa76634a64ac889513d6a3158f" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 32825 } }
I20260806 11:39:08.717023 3318 raft_consensus.cc:348] T 08ed7c66e9024375b7965c1f7f5dbd4a P 08f68b20f34146728d1be1ec93b4a6a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8750dcbab26d4b049421d1db2624af86" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40305 } } peers { permanent_uuid: "08f68b20f34146728d1be1ec93b4a6a2" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 34079 } } peers { permanent_uuid: "359a24fa76634a64ac889513d6a3158f" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 32825 } }
I20260806 11:39:08.717406 3318 raft_consensus.cc:374] T 08ed7c66e9024375b7965c1f7f5dbd4a P 08f68b20f34146728d1be1ec93b4a6a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:39:08.717408 3314 raft_consensus.cc:374] T 08ed7c66e9024375b7965c1f7f5dbd4a P 8750dcbab26d4b049421d1db2624af86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:39:08.717484 3314 raft_consensus.cc:729] T 08ed7c66e9024375b7965c1f7f5dbd4a P 8750dcbab26d4b049421d1db2624af86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8750dcbab26d4b049421d1db2624af86, State: Initialized, Role: FOLLOWER
I20260806 11:39:08.717484 3318 raft_consensus.cc:729] T 08ed7c66e9024375b7965c1f7f5dbd4a P 08f68b20f34146728d1be1ec93b4a6a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08f68b20f34146728d1be1ec93b4a6a2, State: Initialized, Role: FOLLOWER
I20260806 11:39:08.718627 3318 consensus_queue.cc:260] T 08ed7c66e9024375b7965c1f7f5dbd4a P 08f68b20f34146728d1be1ec93b4a6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8750dcbab26d4b049421d1db2624af86" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40305 } } peers { permanent_uuid: "08f68b20f34146728d1be1ec93b4a6a2" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 34079 } } peers { permanent_uuid: "359a24fa76634a64ac889513d6a3158f" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 32825 } }
I20260806 11:39:08.719362 3317 tablet_bootstrap.cc:654] T 08ed7c66e9024375b7965c1f7f5dbd4a P 359a24fa76634a64ac889513d6a3158f: Neither blocks nor log segments found. Creating new log.
I20260806 11:39:08.719952 3318 ts_tablet_manager.cc:1428] T 08ed7c66e9024375b7965c1f7f5dbd4a P 08f68b20f34146728d1be1ec93b4a6a2: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260806 11:39:08.720397 3139 heartbeater.cc:503] Master 127.21.14.254:45429 was elected leader, sending a full tablet report...
I20260806 11:39:08.720480 3317 log.cc:824] T 08ed7c66e9024375b7965c1f7f5dbd4a P 359a24fa76634a64ac889513d6a3158f: Log is configured to *not* fsync() on all Append() calls
I20260806 11:39:08.721350 3314 consensus_queue.cc:260] T 08ed7c66e9024375b7965c1f7f5dbd4a P 8750dcbab26d4b049421d1db2624af86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8750dcbab26d4b049421d1db2624af86" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40305 } } peers { permanent_uuid: "08f68b20f34146728d1be1ec93b4a6a2" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 34079 } } peers { permanent_uuid: "359a24fa76634a64ac889513d6a3158f" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 32825 } }
I20260806 11:39:08.722653 3314 ts_tablet_manager.cc:1428] T 08ed7c66e9024375b7965c1f7f5dbd4a P 8750dcbab26d4b049421d1db2624af86: Time spent starting tablet: real 0.010s user 0.008s sys 0.000s
I20260806 11:39:08.723100 2996 heartbeater.cc:503] Master 127.21.14.254:45429 was elected leader, sending a full tablet report...
I20260806 11:39:08.728417 3317 tablet_bootstrap.cc:492] T 08ed7c66e9024375b7965c1f7f5dbd4a P 359a24fa76634a64ac889513d6a3158f: No bootstrap required, opened a new log
I20260806 11:39:08.728675 3317 ts_tablet_manager.cc:1397] T 08ed7c66e9024375b7965c1f7f5dbd4a P 359a24fa76634a64ac889513d6a3158f: Time spent bootstrapping tablet: real 0.021s user 0.006s sys 0.000s
I20260806 11:39:08.732841 3317 raft_consensus.cc:348] T 08ed7c66e9024375b7965c1f7f5dbd4a P 359a24fa76634a64ac889513d6a3158f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8750dcbab26d4b049421d1db2624af86" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40305 } } peers { permanent_uuid: "08f68b20f34146728d1be1ec93b4a6a2" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 34079 } } peers { permanent_uuid: "359a24fa76634a64ac889513d6a3158f" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 32825 } }
I20260806 11:39:08.733251 3317 raft_consensus.cc:374] T 08ed7c66e9024375b7965c1f7f5dbd4a P 359a24fa76634a64ac889513d6a3158f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:39:08.733310 3317 raft_consensus.cc:729] T 08ed7c66e9024375b7965c1f7f5dbd4a P 359a24fa76634a64ac889513d6a3158f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 359a24fa76634a64ac889513d6a3158f, State: Initialized, Role: FOLLOWER
I20260806 11:39:08.733832 3317 consensus_queue.cc:260] T 08ed7c66e9024375b7965c1f7f5dbd4a P 359a24fa76634a64ac889513d6a3158f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8750dcbab26d4b049421d1db2624af86" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40305 } } peers { permanent_uuid: "08f68b20f34146728d1be1ec93b4a6a2" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 34079 } } peers { permanent_uuid: "359a24fa76634a64ac889513d6a3158f" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 32825 } }
I20260806 11:39:08.735066 3317 ts_tablet_manager.cc:1428] T 08ed7c66e9024375b7965c1f7f5dbd4a P 359a24fa76634a64ac889513d6a3158f: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 11:39:08.735467 3280 heartbeater.cc:503] Master 127.21.14.254:45429 was elected leader, sending a full tablet report...
W20260806 11:39:08.751724 2997 tablet.cc:2369] T 08ed7c66e9024375b7965c1f7f5dbd4a P 8750dcbab26d4b049421d1db2624af86: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 11:39:08 dist-test-slave-7c35 krb5kdc[1683](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016348, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.254@KRBTEST.COM for HTTP/127.21.14.212@KRBTEST.COM
W20260806 11:39:08.790232 3140 tablet.cc:2369] T 08ed7c66e9024375b7965c1f7f5dbd4a P 08f68b20f34146728d1be1ec93b4a6a2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 11:39:08.832866 3325 raft_consensus.cc:482] T 08ed7c66e9024375b7965c1f7f5dbd4a P 359a24fa76634a64ac889513d6a3158f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 11:39:08.833159 3325 raft_consensus.cc:504] T 08ed7c66e9024375b7965c1f7f5dbd4a P 359a24fa76634a64ac889513d6a3158f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8750dcbab26d4b049421d1db2624af86" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40305 } } peers { permanent_uuid: "08f68b20f34146728d1be1ec93b4a6a2" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 34079 } } peers { permanent_uuid: "359a24fa76634a64ac889513d6a3158f" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 32825 } }
I20260806 11:39:08.834780 3325 leader_election.cc:288] T 08ed7c66e9024375b7965c1f7f5dbd4a P 359a24fa76634a64ac889513d6a3158f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8750dcbab26d4b049421d1db2624af86 (127.21.14.193:40305), 08f68b20f34146728d1be1ec93b4a6a2 (127.21.14.194:34079)
I20260806 11:39:08.839936 3310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:08.835483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:39763 (local address 127.21.14.193:40305)
0806 11:39:08.835658 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:08.835663 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:08.835939 (+ 276us) server_negotiation.cc:407] Connection header received
0806 11:39:08.836204 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:08.836208 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:08.836270 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:08.836313 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:39:08.836740 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:08.837353 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:08.838623 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:08.839285 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:08.839443 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:08.839504 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:08.839704 (+ 200us) server_negotiation.cc:299] Negotiation successful
0806 11:39:08.839782 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 11:39:08.839946 3328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:08.835002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.193:40305 (local address 127.21.14.195:39763)
0806 11:39:08.835840 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:08.835857 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 11:39:08.836044 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:39:08.836449 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:39:08.836451 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:39:08.836467 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:39:08.836602 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:08.836607 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:08.837510 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:08.837515 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:08.838468 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:08.838480 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:08.839443 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:08.839447 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:08.839554 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:08.839577 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 11:39:08.839687 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 11:39:08.839774 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":757,"thread_start_us":133,"threads_started":1}
I20260806 11:39:08.840683 2945 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08ed7c66e9024375b7965c1f7f5dbd4a" candidate_uuid: "359a24fa76634a64ac889513d6a3158f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8750dcbab26d4b049421d1db2624af86" is_pre_election: true
I20260806 11:39:08.841068 2945 raft_consensus.cc:2454] T 08ed7c66e9024375b7965c1f7f5dbd4a P 8750dcbab26d4b049421d1db2624af86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 359a24fa76634a64ac889513d6a3158f in term 0.
I20260806 11:39:08.841636 3165 leader_election.cc:302] T 08ed7c66e9024375b7965c1f7f5dbd4a P 359a24fa76634a64ac889513d6a3158f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 359a24fa76634a64ac889513d6a3158f, 8750dcbab26d4b049421d1db2624af86; no voters:
I20260806 11:39:08.841977 3325 raft_consensus.cc:2781] T 08ed7c66e9024375b7965c1f7f5dbd4a P 359a24fa76634a64ac889513d6a3158f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 11:39:08.842079 3325 raft_consensus.cc:482] T 08ed7c66e9024375b7965c1f7f5dbd4a P 359a24fa76634a64ac889513d6a3158f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 11:39:08.842126 3325 raft_consensus.cc:3037] T 08ed7c66e9024375b7965c1f7f5dbd4a P 359a24fa76634a64ac889513d6a3158f [term 0 FOLLOWER]: Advancing to term 1
W20260806 11:39:08.843003 3281 tablet.cc:2369] T 08ed7c66e9024375b7965c1f7f5dbd4a P 359a24fa76634a64ac889513d6a3158f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 11:39:08.843713 3325 raft_consensus.cc:504] T 08ed7c66e9024375b7965c1f7f5dbd4a P 359a24fa76634a64ac889513d6a3158f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8750dcbab26d4b049421d1db2624af86" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40305 } } peers { permanent_uuid: "08f68b20f34146728d1be1ec93b4a6a2" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 34079 } } peers { permanent_uuid: "359a24fa76634a64ac889513d6a3158f" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 32825 } }
I20260806 11:39:08.844184 3325 leader_election.cc:288] T 08ed7c66e9024375b7965c1f7f5dbd4a P 359a24fa76634a64ac889513d6a3158f [CANDIDATE]: Term 1 election: Requested vote from peers 8750dcbab26d4b049421d1db2624af86 (127.21.14.193:40305), 08f68b20f34146728d1be1ec93b4a6a2 (127.21.14.194:34079)
I20260806 11:39:08.844805 2945 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08ed7c66e9024375b7965c1f7f5dbd4a" candidate_uuid: "359a24fa76634a64ac889513d6a3158f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8750dcbab26d4b049421d1db2624af86"
I20260806 11:39:08.844972 2945 raft_consensus.cc:3037] T 08ed7c66e9024375b7965c1f7f5dbd4a P 8750dcbab26d4b049421d1db2624af86 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:39:08.846154 2945 raft_consensus.cc:2454] T 08ed7c66e9024375b7965c1f7f5dbd4a P 8750dcbab26d4b049421d1db2624af86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 359a24fa76634a64ac889513d6a3158f in term 1.
I20260806 11:39:08.846590 3165 leader_election.cc:302] T 08ed7c66e9024375b7965c1f7f5dbd4a P 359a24fa76634a64ac889513d6a3158f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 359a24fa76634a64ac889513d6a3158f, 8750dcbab26d4b049421d1db2624af86; no voters:
I20260806 11:39:08.846943 3325 raft_consensus.cc:2781] T 08ed7c66e9024375b7965c1f7f5dbd4a P 359a24fa76634a64ac889513d6a3158f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:39:08.847258 3325 raft_consensus.cc:686] T 08ed7c66e9024375b7965c1f7f5dbd4a P 359a24fa76634a64ac889513d6a3158f [term 1 LEADER]: Becoming Leader. State: Replica: 359a24fa76634a64ac889513d6a3158f, State: Running, Role: LEADER
I20260806 11:39:08.847662 3325 consensus_queue.cc:237] T 08ed7c66e9024375b7965c1f7f5dbd4a P 359a24fa76634a64ac889513d6a3158f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8750dcbab26d4b049421d1db2624af86" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40305 } } peers { permanent_uuid: "08f68b20f34146728d1be1ec93b4a6a2" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 34079 } } peers { permanent_uuid: "359a24fa76634a64ac889513d6a3158f" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 32825 } }
I20260806 11:39:08.853107 3311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:08.835499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:45363 (local address 127.21.14.194:34079)
0806 11:39:08.835650 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:08.835656 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:08.836145 (+ 489us) server_negotiation.cc:407] Connection header received
0806 11:39:08.837545 (+ 1400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:08.837549 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:08.837608 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:08.837660 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:39:08.849125 (+ 11465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:08.849981 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:08.851629 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:08.852248 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:08.852365 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:08.852424 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:08.852822 (+ 398us) server_negotiation.cc:299] Negotiation successful
0806 11:39:08.852935 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 11:39:08.853627 3329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:08.835107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.194:34079 (local address 127.21.14.195:45363)
0806 11:39:08.836030 (+ 923us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:08.836046 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 11:39:08.837244 (+ 1198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:39:08.838762 (+ 1518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:39:08.838766 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:39:08.838786 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:39:08.848914 (+ 10128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:08.848934 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:08.850169 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:08.850172 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:08.851435 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:08.851449 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:08.852551 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:08.852554 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:08.852662 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:08.852685 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 11:39:08.853346 (+ 661us) client_negotiation.cc:240] Negotiation successful
0806 11:39:08.853441 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":852,"thread_start_us":97,"threads_started":1}
I20260806 11:39:08.853684 2778 catalog_manager.cc:5495] T 08ed7c66e9024375b7965c1f7f5dbd4a P 359a24fa76634a64ac889513d6a3158f reported cstate change: term changed from 0 to 1, leader changed from <none> to 359a24fa76634a64ac889513d6a3158f (127.21.14.195). New cstate: current_term: 1 leader_uuid: "359a24fa76634a64ac889513d6a3158f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8750dcbab26d4b049421d1db2624af86" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "08f68b20f34146728d1be1ec93b4a6a2" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 34079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "359a24fa76634a64ac889513d6a3158f" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 32825 } health_report { overall_health: HEALTHY } } }
I20260806 11:39:08.855225 3084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08ed7c66e9024375b7965c1f7f5dbd4a" candidate_uuid: "359a24fa76634a64ac889513d6a3158f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08f68b20f34146728d1be1ec93b4a6a2" is_pre_election: true
I20260806 11:39:08.855638 3084 raft_consensus.cc:2454] T 08ed7c66e9024375b7965c1f7f5dbd4a P 08f68b20f34146728d1be1ec93b4a6a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 359a24fa76634a64ac889513d6a3158f in term 0.
I20260806 11:39:08.856063 3079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08ed7c66e9024375b7965c1f7f5dbd4a" candidate_uuid: "359a24fa76634a64ac889513d6a3158f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08f68b20f34146728d1be1ec93b4a6a2"
I20260806 11:39:08.856209 3079 raft_consensus.cc:3037] T 08ed7c66e9024375b7965c1f7f5dbd4a P 08f68b20f34146728d1be1ec93b4a6a2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:39:08.857623 3079 raft_consensus.cc:2454] T 08ed7c66e9024375b7965c1f7f5dbd4a P 08f68b20f34146728d1be1ec93b4a6a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 359a24fa76634a64ac889513d6a3158f in term 1.
I20260806 11:39:08.872669 3313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:08.866955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37172 (local address 127.21.14.195:32825)
0806 11:39:08.867122 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:08.867127 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:08.867146 (+ 19us) server_negotiation.cc:407] Connection header received
0806 11:39:08.867245 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:08.867249 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:08.867323 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:08.867449 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 11:39:08.867906 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:08.868732 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:08.870134 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:08.870484 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:08.870606 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:08.871131 (+ 525us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 11:39:08.871763 (+ 632us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 11:39:08.871891 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:08.872396 (+ 505us) server_negotiation.cc:299] Negotiation successful
0806 11:39:08.872470 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 11:39:08.888118 3079 raft_consensus.cc:1261] T 08ed7c66e9024375b7965c1f7f5dbd4a P 08f68b20f34146728d1be1ec93b4a6a2 [term 1 FOLLOWER]: Refusing update from remote peer 359a24fa76634a64ac889513d6a3158f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 11:39:08.888147 2945 raft_consensus.cc:1261] T 08ed7c66e9024375b7965c1f7f5dbd4a P 8750dcbab26d4b049421d1db2624af86 [term 1 FOLLOWER]: Refusing update from remote peer 359a24fa76634a64ac889513d6a3158f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 11:39:08.888985 3325 consensus_queue.cc:1035] T 08ed7c66e9024375b7965c1f7f5dbd4a P 359a24fa76634a64ac889513d6a3158f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8750dcbab26d4b049421d1db2624af86" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 40305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 11:39:08.889534 3330 consensus_queue.cc:1035] T 08ed7c66e9024375b7965c1f7f5dbd4a P 359a24fa76634a64ac889513d6a3158f [LEADER]: Connected to new peer: Peer: permanent_uuid: "08f68b20f34146728d1be1ec93b4a6a2" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 34079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 11:39:08.902392 3333 mvcc.cc:205] Tried to move back new op lower bound from 7315522965032177664 to 7315522964888813568. Current Snapshot: MvccSnapshot[applied={T|T < 7315522965032177664}]
I20260806 11:39:08.918949 2778 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51066:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 11:39:08.919317 2778 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51066:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 11:39:08.922809 2778 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6cb55d7083d2414f8fd77a5beca4c46b: Sending DeleteTablet for 3 replicas of tablet 08ed7c66e9024375b7965c1f7f5dbd4a
I20260806 11:39:08.924288 2927 tablet_service.cc:1514] Processing DeleteTablet for tablet 08ed7c66e9024375b7965c1f7f5dbd4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:39:08 UTC) from {username='kudu', principal='kudu/127.21.14.254@KRBTEST.COM'} at 127.0.0.1:51026
I20260806 11:39:08.924271 3068 tablet_service.cc:1514] Processing DeleteTablet for tablet 08ed7c66e9024375b7965c1f7f5dbd4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:39:08 UTC) from {username='kudu', principal='kudu/127.21.14.254@KRBTEST.COM'} at 127.0.0.1:32830
I20260806 11:39:08.925328 3212 tablet_service.cc:1514] Processing DeleteTablet for tablet 08ed7c66e9024375b7965c1f7f5dbd4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:39:08 UTC) from {username='kudu', principal='kudu/127.21.14.254@KRBTEST.COM'} at 127.0.0.1:37160
I20260806 11:39:08.926723 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 2825
I20260806 11:39:08.927973 3343 tablet_replica.cc:330] T 08ed7c66e9024375b7965c1f7f5dbd4a P 08f68b20f34146728d1be1ec93b4a6a2: stopping tablet replica
I20260806 11:39:08.931103 3344 tablet_replica.cc:330] T 08ed7c66e9024375b7965c1f7f5dbd4a P 359a24fa76634a64ac889513d6a3158f: stopping tablet replica
I20260806 11:39:08.931525 3344 raft_consensus.cc:2229] T 08ed7c66e9024375b7965c1f7f5dbd4a P 359a24fa76634a64ac889513d6a3158f [term 1 LEADER]: Raft consensus shutting down.
I20260806 11:39:08.931962 3343 raft_consensus.cc:2229] T 08ed7c66e9024375b7965c1f7f5dbd4a P 08f68b20f34146728d1be1ec93b4a6a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 11:39:08.932034 3344 raft_consensus.cc:2258] T 08ed7c66e9024375b7965c1f7f5dbd4a P 359a24fa76634a64ac889513d6a3158f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 11:39:08.933782 3344 ts_tablet_manager.cc:1905] T 08ed7c66e9024375b7965c1f7f5dbd4a P 359a24fa76634a64ac889513d6a3158f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 11:39:08.934159 3343 pending_rounds.cc:62] T 08ed7c66e9024375b7965c1f7f5dbd4a P 08f68b20f34146728d1be1ec93b4a6a2: Trying to abort 2 pending ops.
I20260806 11:39:08.934329 3343 pending_rounds.cc:68] T 08ed7c66e9024375b7965c1f7f5dbd4a P 08f68b20f34146728d1be1ec93b4a6a2: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7315522965032177664 op_type: WRITE_OP write_request { tablet_id: "08ed7c66e9024375b7965c1f7f5dbd4a" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "val" type: INT32 is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\003\00090\000\0001\324\000\000" indirect_data: "<redacted>""" } external_consistency_mode: CLIENT_PROPAGATED authz_token { token_data: "\010\210\347\321\323\006\"8\n\ntest-admin\022*\n e07d03ce03ad4220bd7d2495a88b0332\020\001\030\001 \001(\001" signature: "<redacted>""or\223\263=\341\013\263,\014\245\204\254[\366\031D\302\323\336\323\232\372\201\n\352/:N\206\"\317\232\262\241\270\316\332T\037*Gr5\346G\315@Fe\3573^d\372M\214\311\355\362\217\"\201\205" signing_key_seq_num: 0 } } request_id { client_id: "df69f62f217d4ecb9de2ec3c50701f8f" seq_no: 0 first_incomplete_seq_no: 0 attempt_no: 0 }
I20260806 11:39:08.937152 3344 ts_tablet_manager.cc:1918] T 08ed7c66e9024375b7965c1f7f5dbd4a P 359a24fa76634a64ac889513d6a3158f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 11:39:08.937301 3344 log.cc:1196] T 08ed7c66e9024375b7965c1f7f5dbd4a P 359a24fa76634a64ac889513d6a3158f: Deleting WAL directory at /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786016169045816-21563-0/minicluster-data/ts-2/wal/wals/08ed7c66e9024375b7965c1f7f5dbd4a
I20260806 11:39:08.937672 3344 ts_tablet_manager.cc:1939] T 08ed7c66e9024375b7965c1f7f5dbd4a P 359a24fa76634a64ac889513d6a3158f: Deleting consensus metadata
I20260806 11:39:08.938503 3343 pending_rounds.cc:68] T 08ed7c66e9024375b7965c1f7f5dbd4a P 08f68b20f34146728d1be1ec93b4a6a2: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7315522964888813568 op_type: NO_OP noop_request { }
I20260806 11:39:08.938678 3343 raft_consensus.cc:2866] T 08ed7c66e9024375b7965c1f7f5dbd4a P 08f68b20f34146728d1be1ec93b4a6a2 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260806 11:39:08.938787 3343 raft_consensus.cc:2258] T 08ed7c66e9024375b7965c1f7f5dbd4a P 08f68b20f34146728d1be1ec93b4a6a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 11:39:08.939163 2764 catalog_manager.cc:4841] TS 359a24fa76634a64ac889513d6a3158f (127.21.14.195:32825): tablet 08ed7c66e9024375b7965c1f7f5dbd4a (table test-table [id=e07d03ce03ad4220bd7d2495a88b0332]) successfully deleted
W20260806 11:39:08.941277 2767 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.14.193:40305 (error 108)
I20260806 11:39:08.942464 3312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:08.942122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.193:40305 (local address 127.0.0.1:51030)
0806 11:39:08.942258 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:08.942348 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.193:40305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260806 11:39:08.942620 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 3000
W20260806 11:39:08.942876 2767 catalog_manager.cc:4584] TS 8750dcbab26d4b049421d1db2624af86 (127.21.14.193:40305): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 08ed7c66e9024375b7965c1f7f5dbd4a: Network error: Client connection negotiation failed: client connection to 127.21.14.193:40305: connect: Connection refused (error 111)
I20260806 11:39:08.955277 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 3142
I20260806 11:39:08.955533 3312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:08.955098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.194:34079 (local address 127.0.0.1:32834)
0806 11:39:08.955343 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:08.955415 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.194:34079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142}
I20260806 11:39:08.965283 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 2744
I20260806 11:39:08.975034 21563 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 11:39:09.321251 21563 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 11:39:09.321381 21563 mini_ranger.cc:66] Stopping Ranger...
I20260806 11:39:09.667810 21563 mini_ranger.cc:68] Stopped Ranger
2026-08-06 11:39:09.668 UTC [1717] LOG: received smart shutdown request
2026-08-06 11:39:09.674 UTC [1717] LOG: background worker "logical replication launcher" (PID 1728) exited with exit code 1
2026-08-06 11:39:09.674 UTC [1722] LOG: shutting down
2026-08-06 11:39:09.925 UTC [1717] LOG: database system is shut down
2026-08-06T11:39:09Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (54957 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 11:39:10 dist-test-slave-7c35 krb5kdc[3355](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 11:39:10 dist-test-slave-7c35 krb5kdc[3355](info): set up 2 sockets
Aug 06 11:39:10 dist-test-slave-7c35 krb5kdc[3355](info): commencing operation
krb5kdc: starting...
W20260806 11:39:11.188077 21563 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Aug 06 11:39:11 dist-test-slave-7c35 krb5kdc[3355](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016351, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T11:39:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T11:39:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.14.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab.
Entry for principal kudu/127.21.14.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab.
I20260806 11:39:11.338533 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:33293
--webserver_interface=127.21.14.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab
--principal=kudu/127.21.14.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:42845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.14.254:33293,127.21.14.253:32951,127.21.14.252:34411
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:39:11.462316 3368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:39:11.462706 3368 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:39:11.462790 3368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:39:11.467365 3368 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 11:39:11.467496 3368 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:39:11.467537 3368 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:39:11.467578 3368 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 11:39:11.467614 3368 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 11:39:11.473520 3368 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:42845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.14.254:33293,127.21.14.253:32951,127.21.14.252:34411
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab
--principal=kudu/127.21.14.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:33293
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.14.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3368
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:39:11.474803 3368 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:39:11.476089 3368 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:39:11.485450 3373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:39:11.485522 3368 server_base.cc:1034] running on GCE node
W20260806 11:39:11.485463 3374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:39:11.485476 3376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:39:11.486405 3368 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:39:11.487738 3368 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:39:11.488940 3368 hybrid_clock.cc:648] HybridClock initialized: now 1786016351488908 us; error 49 us; skew 500 ppm
Aug 06 11:39:11 dist-test-slave-7c35 krb5kdc[3355](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016351, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:39:11.492223 3368 init.cc:377] Logged in from keytab as kudu/127.21.14.254@KRBTEST.COM (short username kudu)
I20260806 11:39:11.493257 3368 webserver.cc:460] Webserver started at http://127.21.14.254:39907/ using document root <none> and password file <none>
I20260806 11:39:11.493885 3368 fs_manager.cc:359] Metadata directory not provided
I20260806 11:39:11.493952 3368 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:39:11.494190 3368 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:39:11.496498 3368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/minicluster-data/master-0/data/instance:
uuid: "4e36dac8bf0544e8bff9445954b5ba1b"
format_stamp: "Formatted at 2026-08-06 11:39:11 on dist-test-slave-7c35"
server_key: "2545afebe975a3f7eda18a8dc8f99f02"
server_key_iv: "3ce463a394815cba4417b6599995b5e3"
server_key_version: "encryptionkey@0"
I20260806 11:39:11.497128 3368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/minicluster-data/master-0/wal/instance:
uuid: "4e36dac8bf0544e8bff9445954b5ba1b"
format_stamp: "Formatted at 2026-08-06 11:39:11 on dist-test-slave-7c35"
server_key: "2545afebe975a3f7eda18a8dc8f99f02"
server_key_iv: "3ce463a394815cba4417b6599995b5e3"
server_key_version: "encryptionkey@0"
I20260806 11:39:11.501364 3368 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 11:39:11.504287 3383 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:39:11.505853 3368 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 11:39:11.506033 3368 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/minicluster-data/master-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/minicluster-data/master-0/wal
uuid: "4e36dac8bf0544e8bff9445954b5ba1b"
format_stamp: "Formatted at 2026-08-06 11:39:11 on dist-test-slave-7c35"
server_key: "2545afebe975a3f7eda18a8dc8f99f02"
server_key_iv: "3ce463a394815cba4417b6599995b5e3"
server_key_version: "encryptionkey@0"
I20260806 11:39:11.506175 3368 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:39:11.533365 3368 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:39:11.536818 3368 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:39:11.537045 3368 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:39:11.545121 3368 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.254:33293
I20260806 11:39:11.545146 3439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.254:33293 every 8 connection(s)
I20260806 11:39:11.546229 3368 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
I20260806 11:39:11.547664 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 3368
I20260806 11:39:11.547827 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/minicluster-data/master-0/wal/instance
I20260806 11:39:11.548179 21563 external_mini_cluster.cc:1428] Setting key 0f6f85c1c35f89ddc78ba0a7e2d3b528
I20260806 11:39:11.550099 3441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:39:11.554603 3441 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.14.254" port: 33293 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.21.14.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.253@KRBTEST.COM" created.
Aug 06 11:39:11 dist-test-slave-7c35 krb5kdc[3355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016351, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.254@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:39:11.573140 3445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:11.557446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:33293 (local address 127.0.0.1:41074)
0806 11:39:11.558058 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:11.558114 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 11:39:11.559200 (+ 1086us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:39:11.560052 (+ 852us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:39:11.560057 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:39:11.560414 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:39:11.560707 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:11.560754 (+ 47us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:11.562661 (+ 1907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:11.562665 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:11.563550 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:11.563561 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:11.564536 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:11.564540 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:11.564696 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:11.565386 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:39:11.565414 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:39:11.567519 (+ 2105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:39:11.569759 (+ 2240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:39:11.569770 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:39:11.569784 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:39:11.570160 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:39:11.570539 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:39:11.570542 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:39:11.570544 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:39:11.570715 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:39:11.571483 (+ 768us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:39:11.571492 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:39:11.571700 (+ 208us) client_negotiation.cc:769] Sending connection context
0806 11:39:11.571906 (+ 206us) client_negotiation.cc:240] Negotiation successful
0806 11:39:11.572298 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":402,"mutex_wait_us":121,"spinlock_wait_cycles":14720,"thread_start_us":215,"threads_started":1}
I20260806 11:39:11.573138 3444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:11.557446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41074 (local address 127.21.14.254:33293)
0806 11:39:11.557955 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:11.557971 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:11.558272 (+ 301us) server_negotiation.cc:407] Connection header received
0806 11:39:11.559521 (+ 1249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:11.559526 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:11.559765 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:11.559955 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:39:11.560944 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:11.562451 (+ 1507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:11.563687 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:11.564349 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:11.564606 (+ 257us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:11.567711 (+ 3105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:39:11.567717 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:39:11.567730 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:39:11.567755 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:39:11.569620 (+ 1865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:11.570297 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:11.570303 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:11.570304 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:11.570428 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:11.570826 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:11.570829 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:11.570830 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:11.571341 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:39:11.571464 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:11.571997 (+ 533us) server_negotiation.cc:299] Negotiation successful
0806 11:39:11.572215 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":337,"thread_start_us":154,"threads_started":1}
I20260806 11:39:11.575801 3441 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.14.253" port: 32951 } has no permanent_uuid. Determining permanent_uuid...
I20260806 11:39:11.576774 3445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:11.576432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.253:32951 (local address 127.0.0.1:60840)
0806 11:39:11.576613 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:11.576703 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.253:32951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260806 11:39:11.577037 3386 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.14.253:32951: connect: Connection refused (error 111)
I20260806 11:39:11.577983 3445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:11.577731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.253:32951 (local address 127.0.0.1:60848)
0806 11:39:11.577864 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:11.577925 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.253:32951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260806 11:39:11.578173 3441 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.14.253:32951: Network error: Client connection negotiation failed: client connection to 127.21.14.253:32951: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.253.keytab.
Entry for principal kudu/127.21.14.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.253.keytab.
I20260806 11:39:11.582255 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.253:32951
--webserver_interface=127.21.14.253
--webserver_port=0
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.253.keytab
--principal=kudu/127.21.14.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:42845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.14.254:33293,127.21.14.253:32951,127.21.14.252:34411
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 11:39:11.627460 3441 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.14.253" port: 32951 } attempt: 1
I20260806 11:39:11.628487 3445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:11.628151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.253:32951 (local address 127.0.0.1:60852)
0806 11:39:11.628329 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:11.628399 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.253:32951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260806 11:39:11.629438 3445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:11.629164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.253:32951 (local address 127.0.0.1:60864)
0806 11:39:11.629302 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:11.629366 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.253:32951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260806 11:39:11.629623 3441 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.14.253:32951: Network error: Client connection negotiation failed: client connection to 127.21.14.253:32951: connect: Connection refused (error 111)
I20260806 11:39:11.697810 3441 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.14.253" port: 32951 } attempt: 2
I20260806 11:39:11.698827 3445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:11.698468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.253:32951 (local address 127.0.0.1:60866)
0806 11:39:11.698672 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:11.698742 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.253:32951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260806 11:39:11.699734 3445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:11.699470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.253:32951 (local address 127.0.0.1:60874)
0806 11:39:11.699607 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:11.699669 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.253:32951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260806 11:39:11.699910 3441 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.14.253:32951: Network error: Client connection negotiation failed: client connection to 127.21.14.253:32951: connect: Connection refused (error 111)
W20260806 11:39:11.711699 3448 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:39:11.712037 3448 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:39:11.712155 3448 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:39:11.716411 3448 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 11:39:11.716507 3448 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:39:11.716544 3448 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:39:11.716585 3448 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 11:39:11.716615 3448 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 11:39:11.722486 3448 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:42845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.21.14.254:33293,127.21.14.253:32951,127.21.14.252:34411
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.253.keytab
--principal=kudu/127.21.14.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.253:32951
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.21.14.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3448
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:39:11.723791 3448 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:39:11.724920 3448 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:39:11.734097 3457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:39:11.734102 3455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:39:11.734134 3454 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:39:11.734573 3448 server_base.cc:1034] running on GCE node
I20260806 11:39:11.735199 3448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:39:11.736356 3448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:39:11.737630 3448 hybrid_clock.cc:648] HybridClock initialized: now 1786016351737615 us; error 30 us; skew 500 ppm
Aug 06 11:39:11 dist-test-slave-7c35 krb5kdc[3355](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016351, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:39:11.740803 3448 init.cc:377] Logged in from keytab as kudu/127.21.14.253@KRBTEST.COM (short username kudu)
I20260806 11:39:11.741760 3448 webserver.cc:460] Webserver started at http://127.21.14.253:42025/ using document root <none> and password file <none>
I20260806 11:39:11.742349 3448 fs_manager.cc:359] Metadata directory not provided
I20260806 11:39:11.742417 3448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:39:11.742604 3448 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:39:11.744870 3448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/minicluster-data/master-1/data/instance:
uuid: "ce883cfa54124f00aaab713b62eeb554"
format_stamp: "Formatted at 2026-08-06 11:39:11 on dist-test-slave-7c35"
server_key: "a7e3efb9d5023ca2d61d23a752838e56"
server_key_iv: "92398c7f754320672f03a4e45b5264ad"
server_key_version: "encryptionkey@0"
I20260806 11:39:11.745437 3448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/minicluster-data/master-1/wal/instance:
uuid: "ce883cfa54124f00aaab713b62eeb554"
format_stamp: "Formatted at 2026-08-06 11:39:11 on dist-test-slave-7c35"
server_key: "a7e3efb9d5023ca2d61d23a752838e56"
server_key_iv: "92398c7f754320672f03a4e45b5264ad"
server_key_version: "encryptionkey@0"
I20260806 11:39:11.749691 3448 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260806 11:39:11.752643 3465 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:39:11.754103 3448 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 11:39:11.754240 3448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/minicluster-data/master-1/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/minicluster-data/master-1/wal
uuid: "ce883cfa54124f00aaab713b62eeb554"
format_stamp: "Formatted at 2026-08-06 11:39:11 on dist-test-slave-7c35"
server_key: "a7e3efb9d5023ca2d61d23a752838e56"
server_key_iv: "92398c7f754320672f03a4e45b5264ad"
server_key_version: "encryptionkey@0"
I20260806 11:39:11.754349 3448 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:39:11.768656 3448 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:39:11.772176 3448 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:39:11.772356 3448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:39:11.780257 3448 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.253:32951
I20260806 11:39:11.780287 3519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.253:32951 every 8 connection(s)
I20260806 11:39:11.781402 3448 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/minicluster-data/master-1/data/info.pb
I20260806 11:39:11.784690 3520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:39:11.789157 3520 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.14.254" port: 33293 } has no permanent_uuid. Determining permanent_uuid...
I20260806 11:39:11.791066 3441 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.14.253" port: 32951 } attempt: 3
I20260806 11:39:11.791155 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 3448
I20260806 11:39:11.791307 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/minicluster-data/master-1/wal/instance
I20260806 11:39:11.791721 21563 external_mini_cluster.cc:1428] Setting key 8dc9c593ff281688fc37098d78a9a47c
Aug 06 11:39:11 dist-test-slave-7c35 krb5kdc[3355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016351, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.254@KRBTEST.COM for kudu/127.21.14.253@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.14.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.252@KRBTEST.COM" created.
Aug 06 11:39:11 dist-test-slave-7c35 krb5kdc[3355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016351, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.253@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:39:11.806284 3445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:11.791617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.253:32951 (local address 127.0.0.1:60878)
0806 11:39:11.791744 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:11.791759 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 11:39:11.791862 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:39:11.794607 (+ 2745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:39:11.794612 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:39:11.794886 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:39:11.795068 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:11.795076 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:11.796676 (+ 1600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:11.796682 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:11.797487 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:11.797502 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:11.798896 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:11.798901 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:11.799066 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:11.799720 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:39:11.799740 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:39:11.801414 (+ 1674us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:39:11.803801 (+ 2387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:39:11.803805 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:39:11.803806 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:39:11.804038 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:39:11.804599 (+ 561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:39:11.804603 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:39:11.804604 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:39:11.804725 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:39:11.805644 (+ 919us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:39:11.805649 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:39:11.805786 (+ 137us) client_negotiation.cc:769] Sending connection context
0806 11:39:11.805878 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 11:39:11.806062 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80,"mutex_wait_us":3}
I20260806 11:39:11.808154 3523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:11.792034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60878 (local address 127.21.14.253:32951)
0806 11:39:11.792696 (+ 662us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:11.792711 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:11.792761 (+ 50us) server_negotiation.cc:407] Connection header received
0806 11:39:11.794036 (+ 1275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:11.794072 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:11.794397 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:11.794601 (+ 204us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:39:11.795236 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:11.796534 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:11.797659 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:11.798696 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:11.798909 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:11.801628 (+ 2719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:39:11.801637 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:39:11.801655 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:39:11.801694 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:39:11.803660 (+ 1966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:11.804154 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:11.804167 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:11.804173 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:11.804304 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:11.804860 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:11.804864 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:11.804865 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:11.805486 (+ 621us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:39:11.806352 (+ 866us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:11.806563 (+ 211us) server_negotiation.cc:299] Negotiation successful
0806 11:39:11.806979 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":99,"server-negotiator.queue_time_us":520,"spinlock_wait_cycles":11008,"thread_start_us":179,"threads_started":1}
I20260806 11:39:11.810479 3441 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.14.252" port: 34411 } has no permanent_uuid. Determining permanent_uuid...
I20260806 11:39:11.810567 3522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:11.792026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:33293 (local address 127.0.0.1:41082)
0806 11:39:11.792596 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:11.792653 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 11:39:11.794053 (+ 1400us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:39:11.795021 (+ 968us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:39:11.795028 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:39:11.795555 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:39:11.795777 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:11.795794 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:11.796992 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:11.796996 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:11.798421 (+ 1425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:11.798435 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:11.799740 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:11.799747 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:11.799890 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:11.801138 (+ 1248us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:39:11.801159 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:39:11.806145 (+ 4986us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:39:11.808452 (+ 2307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:39:11.808465 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:39:11.808472 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:39:11.808750 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:39:11.809077 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:39:11.809080 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:39:11.809082 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:39:11.809204 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:39:11.809753 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:39:11.809760 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:39:11.809897 (+ 137us) client_negotiation.cc:769] Sending connection context
0806 11:39:11.810050 (+ 153us) client_negotiation.cc:240] Negotiation successful
0806 11:39:11.810360 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"mutex_wait_us":152,"thread_start_us":162,"threads_started":1}
I20260806 11:39:11.811425 3445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:11.811044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.252:34411 (local address 127.0.0.1:36912)
0806 11:39:11.811262 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:11.811323 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.252:34411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121}
I20260806 11:39:11.811450 3444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:11.791522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41082 (local address 127.21.14.254:33293)
0806 11:39:11.791665 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:11.791670 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:11.792751 (+ 1081us) server_negotiation.cc:407] Connection header received
0806 11:39:11.794378 (+ 1627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:11.794382 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:11.794442 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:11.794549 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:39:11.795973 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:11.796831 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:11.798579 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:11.799166 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:11.799317 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:11.806319 (+ 7002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:39:11.806326 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:39:11.806331 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:39:11.806368 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:39:11.808313 (+ 1945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:11.808869 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:11.808873 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:11.808874 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:11.808972 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:11.809332 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:11.809338 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:11.809341 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:11.809624 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:39:11.809741 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:11.811030 (+ 1289us) server_negotiation.cc:299] Negotiation successful
0806 11:39:11.811194 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":54,"server-negotiator.queue_time_us":58}
I20260806 11:39:11.812638 3445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:11.812347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.252:34411 (local address 127.0.0.1:36922)
0806 11:39:11.812506 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:11.812582 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.252:34411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
W20260806 11:39:11.812897 3441 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.14.252:34411: Network error: Client connection negotiation failed: client connection to 127.21.14.252:34411: connect: Connection refused (error 111)
I20260806 11:39:11.812954 3520 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.14.253" port: 32951 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 11:39:11 dist-test-slave-7c35 krb5kdc[3355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016351, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.253@KRBTEST.COM for kudu/127.21.14.253@KRBTEST.COM
I20260806 11:39:11.825167 3522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:11.813466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.253:32951 (local address 127.0.0.1:60884)
0806 11:39:11.813581 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:11.813597 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 11:39:11.813688 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:39:11.813913 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:39:11.813915 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:39:11.814108 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:39:11.814206 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:11.814212 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:11.815004 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:11.815007 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:11.815540 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:11.815548 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:11.816017 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:11.816020 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:11.816119 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:11.816716 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:39:11.816729 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:39:11.820655 (+ 3926us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:39:11.823296 (+ 2641us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:39:11.823300 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:39:11.823302 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:39:11.823541 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:39:11.823858 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:39:11.823862 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:39:11.823864 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:39:11.823982 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:39:11.824488 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:39:11.824490 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:39:11.824672 (+ 182us) client_negotiation.cc:769] Sending connection context
0806 11:39:11.824789 (+ 117us) client_negotiation.cc:240] Negotiation successful
0806 11:39:11.824960 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 11:39:11.825232 3523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:11.813546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60884 (local address 127.21.14.253:32951)
0806 11:39:11.813651 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:11.813655 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:11.813668 (+ 13us) server_negotiation.cc:407] Connection header received
0806 11:39:11.813770 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:11.813773 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:11.813816 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:11.813901 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:39:11.814324 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:11.814833 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:11.815665 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:11.815900 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:11.816013 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:11.820888 (+ 4875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:39:11.820893 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:39:11.820895 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:39:11.820927 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:39:11.822821 (+ 1894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:11.823668 (+ 847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:11.823671 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:11.823673 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:11.823760 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:11.824118 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:11.824121 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:11.824122 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:11.824345 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:39:11.824451 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:11.824859 (+ 408us) server_negotiation.cc:299] Negotiation successful
0806 11:39:11.825057 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":58,"server-negotiator.queue_time_us":42}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.252.keytab.
Entry for principal kudu/127.21.14.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.252.keytab.
I20260806 11:39:11.826378 3520 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.14.252" port: 34411 } has no permanent_uuid. Determining permanent_uuid...
I20260806 11:39:11.827030 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.252:34411
--webserver_interface=127.21.14.252
--webserver_port=0
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.252.keytab
--principal=kudu/127.21.14.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:42845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.14.254:33293,127.21.14.253:32951,127.21.14.252:34411
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 11:39:11.827396 3522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:11.827022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.252:34411 (local address 127.0.0.1:36924)
0806 11:39:11.827177 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:11.827311 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.252:34411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260806 11:39:11.828284 3467 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.14.252:34411: connect: Connection refused (error 111)
I20260806 11:39:11.829442 3522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:11.829128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.252:34411 (local address 127.0.0.1:36926)
0806 11:39:11.829280 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:11.829348 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.252:34411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260806 11:39:11.829689 3520 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.14.252:34411: Network error: Client connection negotiation failed: client connection to 127.21.14.252:34411: connect: Connection refused (error 111)
I20260806 11:39:11.844062 3441 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.14.252" port: 34411 } attempt: 1
I20260806 11:39:11.845039 3445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:11.844694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.252:34411 (local address 127.0.0.1:36942)
0806 11:39:11.844869 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:11.844937 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.252:34411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260806 11:39:11.845822 3445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:11.845588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.252:34411 (local address 127.0.0.1:36952)
0806 11:39:11.845704 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:11.845761 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.252:34411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260806 11:39:11.845975 3441 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.14.252:34411: Network error: Client connection negotiation failed: client connection to 127.21.14.252:34411: connect: Connection refused (error 111)
I20260806 11:39:11.879032 3520 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.14.252" port: 34411 } attempt: 1
I20260806 11:39:11.880040 3522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:11.879705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.252:34411 (local address 127.0.0.1:36960)
0806 11:39:11.879876 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:11.879952 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.252:34411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260806 11:39:11.880894 3522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:11.880651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.252:34411 (local address 127.0.0.1:36966)
0806 11:39:11.880771 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:11.880828 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.252:34411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260806 11:39:11.881067 3520 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.14.252:34411: Network error: Client connection negotiation failed: client connection to 127.21.14.252:34411: connect: Connection refused (error 111)
I20260806 11:39:11.921178 3441 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.14.252" port: 34411 } attempt: 2
I20260806 11:39:11.922192 3445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:11.921837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.252:34411 (local address 127.0.0.1:36970)
0806 11:39:11.922021 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:11.922102 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.252:34411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260806 11:39:11.923149 3445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:11.922897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.252:34411 (local address 127.0.0.1:36984)
0806 11:39:11.923024 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:11.923084 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.252:34411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260806 11:39:11.923367 3441 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.14.252:34411: Network error: Client connection negotiation failed: client connection to 127.21.14.252:34411: connect: Connection refused (error 111)
I20260806 11:39:11.949250 3520 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.14.252" port: 34411 } attempt: 2
I20260806 11:39:11.950215 3522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:11.949873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.252:34411 (local address 127.0.0.1:37000)
0806 11:39:11.950040 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:11.950131 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.252:34411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260806 11:39:11.951124 3522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:11.950906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.252:34411 (local address 127.0.0.1:37014)
0806 11:39:11.951007 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:11.951060 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.252:34411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260806 11:39:11.951279 3520 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.14.252:34411: Network error: Client connection negotiation failed: client connection to 127.21.14.252:34411: connect: Connection refused (error 111)
W20260806 11:39:11.951489 3527 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:39:11.951807 3527 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:39:11.951910 3527 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:39:11.956979 3527 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 11:39:11.957065 3527 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:39:11.957086 3527 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:39:11.957105 3527 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 11:39:11.957155 3527 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 11:39:11.963292 3527 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:42845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.21.14.254:33293,127.21.14.253:32951,127.21.14.252:34411
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.252.keytab
--principal=kudu/127.21.14.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.252:34411
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.21.14.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3527
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:39:11.964578 3527 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:39:11.965715 3527 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:39:11.975512 3533 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:39:11.975512 3535 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:39:11.975519 3537 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:39:11.975891 3527 server_base.cc:1034] running on GCE node
I20260806 11:39:11.976547 3527 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:39:11.977864 3527 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:39:11.979046 3527 hybrid_clock.cc:648] HybridClock initialized: now 1786016351979022 us; error 43 us; skew 500 ppm
Aug 06 11:39:11 dist-test-slave-7c35 krb5kdc[3355](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016351, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:39:11.982194 3527 init.cc:377] Logged in from keytab as kudu/127.21.14.252@KRBTEST.COM (short username kudu)
I20260806 11:39:11.983322 3527 webserver.cc:460] Webserver started at http://127.21.14.252:40647/ using document root <none> and password file <none>
I20260806 11:39:11.983969 3527 fs_manager.cc:359] Metadata directory not provided
I20260806 11:39:11.984071 3527 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:39:11.984320 3527 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:39:11.986791 3527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/minicluster-data/master-2/data/instance:
uuid: "87d10c23a1d0413a8c9d55caca6811ef"
format_stamp: "Formatted at 2026-08-06 11:39:11 on dist-test-slave-7c35"
server_key: "e2ac3d645a676464f860d24fe5def9cd"
server_key_iv: "fddaee7b11b641c7b2edf248b191b73d"
server_key_version: "encryptionkey@0"
I20260806 11:39:11.987550 3527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/minicluster-data/master-2/wal/instance:
uuid: "87d10c23a1d0413a8c9d55caca6811ef"
format_stamp: "Formatted at 2026-08-06 11:39:11 on dist-test-slave-7c35"
server_key: "e2ac3d645a676464f860d24fe5def9cd"
server_key_iv: "fddaee7b11b641c7b2edf248b191b73d"
server_key_version: "encryptionkey@0"
I20260806 11:39:11.991930 3527 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 11:39:11.994752 3545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:39:11.996291 3527 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 11:39:11.996418 3527 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/minicluster-data/master-2/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/minicluster-data/master-2/wal
uuid: "87d10c23a1d0413a8c9d55caca6811ef"
format_stamp: "Formatted at 2026-08-06 11:39:11 on dist-test-slave-7c35"
server_key: "e2ac3d645a676464f860d24fe5def9cd"
server_key_iv: "fddaee7b11b641c7b2edf248b191b73d"
server_key_version: "encryptionkey@0"
I20260806 11:39:11.996534 3527 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:39:12.016698 3527 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:39:12.020251 3527 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:39:12.020432 3527 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:39:12.022573 3441 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.14.252" port: 34411 } attempt: 3
I20260806 11:39:12.023510 3445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:12.023129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.252:34411 (local address 127.0.0.1:37030)
0806 11:39:12.023319 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:12.023398 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.252:34411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260806 11:39:12.024585 3445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:12.024285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.252:34411 (local address 127.0.0.1:37044)
0806 11:39:12.024437 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:12.024502 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.252:34411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260806 11:39:12.024806 3441 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.14.252:34411: Network error: Client connection negotiation failed: client connection to 127.21.14.252:34411: connect: Connection refused (error 111)
I20260806 11:39:12.028867 3527 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.252:34411
I20260806 11:39:12.028892 3599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.252:34411 every 8 connection(s)
I20260806 11:39:12.029903 3527 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/minicluster-data/master-2/data/info.pb
I20260806 11:39:12.032994 3600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:39:12.036307 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 3527
I20260806 11:39:12.036475 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786016169045816-21563-0/minicluster-data/master-2/wal/instance
I20260806 11:39:12.036779 21563 external_mini_cluster.cc:1428] Setting key c886174e704d4e4ed24af865cff4d3e7
I20260806 11:39:12.037595 3600 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.14.254" port: 33293 } has no permanent_uuid. Determining permanent_uuid...
I20260806 11:39:12.042474 3520 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.14.252" port: 34411 } attempt: 3
Aug 06 11:39:12 dist-test-slave-7c35 krb5kdc[3355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016351, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.252@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
Aug 06 11:39:12 dist-test-slave-7c35 krb5kdc[3355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016351, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.14.252@KRBTEST.COM
Aug 06 11:39:12 dist-test-slave-7c35 krb5kdc[3355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016351, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.253@KRBTEST.COM for kudu/127.21.14.252@KRBTEST.COM
I20260806 11:39:12.053198 3444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:12.039736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41096 (local address 127.21.14.254:33293)
0806 11:39:12.039878 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:12.039882 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:12.040168 (+ 286us) server_negotiation.cc:407] Connection header received
0806 11:39:12.040699 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:12.040703 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:12.040758 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:12.040856 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:39:12.042003 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:12.042612 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:12.044603 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:12.044955 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:12.045074 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:12.048477 (+ 3403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:39:12.048485 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:39:12.048490 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:39:12.048524 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:39:12.050182 (+ 1658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:12.050984 (+ 802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:12.050989 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:12.050991 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:12.051092 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:12.051482 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:12.051486 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:12.051487 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:12.051790 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:39:12.051917 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:12.052837 (+ 920us) server_negotiation.cc:299] Negotiation successful
0806 11:39:12.053019 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 11:39:12.053817 3522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:12.043021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.252:34411 (local address 127.0.0.1:37068)
0806 11:39:12.043141 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:12.043158 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 11:39:12.043240 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:39:12.043811 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:39:12.043814 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:39:12.044007 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:39:12.044097 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:12.044103 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:12.045138 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:12.045144 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:12.045911 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:12.045927 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:12.046553 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:12.046560 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:12.046658 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:12.047247 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:39:12.047294 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:39:12.048543 (+ 1249us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:39:12.051615 (+ 3072us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:39:12.051619 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:39:12.051621 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:39:12.051903 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:39:12.052356 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:39:12.052360 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:39:12.052361 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:39:12.052463 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:39:12.052944 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:39:12.052951 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:39:12.053062 (+ 111us) client_negotiation.cc:769] Sending connection context
0806 11:39:12.053188 (+ 126us) client_negotiation.cc:240] Negotiation successful
0806 11:39:12.053653 (+ 465us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260806 11:39:12.054782 3603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:12.038470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37056 (local address 127.21.14.252:34411)
0806 11:39:12.039023 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:12.039041 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:12.039079 (+ 38us) server_negotiation.cc:407] Connection header received
0806 11:39:12.039916 (+ 837us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:12.039944 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:12.040466 (+ 522us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:12.040749 (+ 283us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:39:12.041548 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:12.042851 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:12.044178 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:12.044710 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:12.044883 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:12.048365 (+ 3482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:39:12.048377 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:39:12.048398 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:39:12.048434 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:39:12.050724 (+ 2290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:12.051380 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:12.051388 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:12.051391 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:12.051494 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:12.052225 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:12.052229 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:12.052231 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:12.052616 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:39:12.052839 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:12.053421 (+ 582us) server_negotiation.cc:299] Negotiation successful
0806 11:39:12.053739 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":390,"spinlock_wait_cycles":5376,"thread_start_us":148,"threads_started":1}
I20260806 11:39:12.054790 3604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:12.039632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:33293 (local address 127.0.0.1:41096)
0806 11:39:12.040006 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:12.040059 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 11:39:12.040451 (+ 392us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:39:12.040927 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:39:12.040932 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:39:12.041597 (+ 665us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:39:12.041842 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:12.041856 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:12.042757 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:12.042766 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:12.044464 (+ 1698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:12.044475 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:12.045146 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:12.045152 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:12.045288 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:12.045807 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:39:12.045824 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:39:12.048304 (+ 2480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:39:12.050322 (+ 2018us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:39:12.050335 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:39:12.050348 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:39:12.050839 (+ 491us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:39:12.051196 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:39:12.051199 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:39:12.051201 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:39:12.051375 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:39:12.051935 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:39:12.051943 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:39:12.052494 (+ 551us) client_negotiation.cc:769] Sending connection context
0806 11:39:12.053612 (+ 1118us) client_negotiation.cc:240] Negotiation successful
0806 11:39:12.053922 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"mutex_wait_us":106,"thread_start_us":94,"threads_started":1}
I20260806 11:39:12.054790 3605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:12.043068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37068 (local address 127.21.14.252:34411)
0806 11:39:12.043454 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:12.043461 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:12.043471 (+ 10us) server_negotiation.cc:407] Connection header received
0806 11:39:12.043508 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:12.043510 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:12.043545 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:12.043798 (+ 253us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:39:12.044230 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:12.044988 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:12.046058 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:12.046385 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:12.046518 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:12.048725 (+ 2207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:39:12.048728 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:39:12.048730 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:39:12.048751 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:39:12.051499 (+ 2748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:12.052011 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:12.052014 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:12.052015 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:12.052113 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:12.052573 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:12.052577 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:12.052579 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:12.052813 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:39:12.052948 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:12.053319 (+ 371us) server_negotiation.cc:299] Negotiation successful
0806 11:39:12.053861 (+ 542us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":165,"server-negotiator.queue_time_us":326,"thread_start_us":175,"threads_started":1}
I20260806 11:39:12.057219 3600 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.14.253" port: 32951 } has no permanent_uuid. Determining permanent_uuid...
I20260806 11:39:12.060735 3520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce883cfa54124f00aaab713b62eeb554: Bootstrap starting.
Aug 06 11:39:12 dist-test-slave-7c35 krb5kdc[3355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016351, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.252@KRBTEST.COM for kudu/127.21.14.253@KRBTEST.COM
I20260806 11:39:12.063881 3520 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce883cfa54124f00aaab713b62eeb554: Neither blocks nor log segments found. Creating new log.
Aug 06 11:39:12 dist-test-slave-7c35 krb5kdc[3355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016351, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:39:12.064767 3520 log.cc:824] T 00000000000000000000000000000000 P ce883cfa54124f00aaab713b62eeb554: Log is configured to *not* fsync() on all Append() calls
I20260806 11:39:12.067512 3520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce883cfa54124f00aaab713b62eeb554: No bootstrap required, opened a new log
I20260806 11:39:12.069077 3523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:12.057880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60898 (local address 127.21.14.253:32951)
0806 11:39:12.058015 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:12.058019 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:12.058560 (+ 541us) server_negotiation.cc:407] Connection header received
0806 11:39:12.058716 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:12.058722 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:12.058777 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:12.058905 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:39:12.059489 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:12.060003 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:12.061146 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:12.061440 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:12.061502 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:12.065001 (+ 3499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:39:12.065005 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:39:12.065007 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:39:12.065034 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:39:12.067055 (+ 2021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:12.067683 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:12.067687 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:12.067689 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:12.067790 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:12.068133 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:12.068136 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:12.068139 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:12.068351 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:39:12.068468 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:12.068768 (+ 300us) server_negotiation.cc:299] Negotiation successful
0806 11:39:12.068896 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 11:39:12.069521 3604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:12.057804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.253:32951 (local address 127.0.0.1:60898)
0806 11:39:12.058491 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:12.058506 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 11:39:12.058623 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:39:12.059058 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:39:12.059061 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:39:12.059243 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:39:12.059384 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:12.059395 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:12.060133 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:12.060136 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:12.060999 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:12.061010 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:12.061653 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:12.061656 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:12.061777 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:12.062366 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:39:12.062378 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:39:12.064748 (+ 2370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:39:12.067230 (+ 2482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:39:12.067235 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:39:12.067239 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:39:12.067501 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:39:12.067924 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:39:12.067929 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:39:12.067932 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:39:12.068029 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:39:12.068459 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:39:12.068462 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:39:12.068576 (+ 114us) client_negotiation.cc:769] Sending connection context
0806 11:39:12.068694 (+ 118us) client_negotiation.cc:240] Negotiation successful
0806 11:39:12.068877 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":600}
I20260806 11:39:12.069840 3444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:12.057990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41100 (local address 127.21.14.254:33293)
0806 11:39:12.058174 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:12.058179 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:12.058240 (+ 61us) server_negotiation.cc:407] Connection header received
0806 11:39:12.058385 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:12.058387 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:12.058430 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:12.058528 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:39:12.059450 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:12.060199 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:12.061305 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:12.061661 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:12.061771 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:12.065033 (+ 3262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:39:12.065037 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:39:12.065040 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:39:12.065074 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:39:12.067184 (+ 2110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:12.067736 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:12.067743 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:12.067746 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:12.067849 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:12.068201 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:12.068205 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:12.068207 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:12.068441 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:39:12.069412 (+ 971us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:12.069533 (+ 121us) server_negotiation.cc:299] Negotiation successful
0806 11:39:12.069683 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 11:39:12.070636 3600 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.14.252" port: 34411 } has no permanent_uuid. Determining permanent_uuid...
I20260806 11:39:12.073256 3520 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce883cfa54124f00aaab713b62eeb554 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e36dac8bf0544e8bff9445954b5ba1b" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 33293 } } peers { permanent_uuid: "ce883cfa54124f00aaab713b62eeb554" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 32951 } } peers { permanent_uuid: "87d10c23a1d0413a8c9d55caca6811ef" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 34411 } }
I20260806 11:39:12.073576 3520 raft_consensus.cc:374] T 00000000000000000000000000000000 P ce883cfa54124f00aaab713b62eeb554 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:39:12.073640 3520 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce883cfa54124f00aaab713b62eeb554 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce883cfa54124f00aaab713b62eeb554, State: Initialized, Role: FOLLOWER
I20260806 11:39:12.074237 3520 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce883cfa54124f00aaab713b62eeb554 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e36dac8bf0544e8bff9445954b5ba1b" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 33293 } } peers { permanent_uuid: "ce883cfa54124f00aaab713b62eeb554" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 32951 } } peers { permanent_uuid: "87d10c23a1d0413a8c9d55caca6811ef" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 34411 } }
I20260806 11:39:12.075379 3607 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce883cfa54124f00aaab713b62eeb554 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e36dac8bf0544e8bff9445954b5ba1b" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 33293 } } peers { permanent_uuid: "ce883cfa54124f00aaab713b62eeb554" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 32951 } } peers { permanent_uuid: "87d10c23a1d0413a8c9d55caca6811ef" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 34411 } } }
I20260806 11:39:12.075937 3520 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce883cfa54124f00aaab713b62eeb554 [sys.catalog]: configured and running, proceeding with master startup.
Aug 06 11:39:12 dist-test-slave-7c35 krb5kdc[3355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016351, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.252@KRBTEST.COM for kudu/127.21.14.252@KRBTEST.COM
I20260806 11:39:12.075947 3607 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce883cfa54124f00aaab713b62eeb554 [sys.catalog]: This master's current role is: FOLLOWER
Aug 06 11:39:12 dist-test-slave-7c35 krb5kdc[3355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016351, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.14.253@KRBTEST.COM
I20260806 11:39:12.082414 3604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:12.071204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.252:34411 (local address 127.0.0.1:37078)
0806 11:39:12.071350 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:12.071363 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 11:39:12.071455 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:39:12.071804 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:39:12.071810 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:39:12.072024 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:39:12.072139 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:12.072148 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:12.072926 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:12.072930 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:12.073777 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:12.073788 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:12.074499 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:12.074503 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:12.074600 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:12.075681 (+ 1081us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:39:12.075700 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:39:12.078234 (+ 2534us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:39:12.080663 (+ 2429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:39:12.080666 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:39:12.080668 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:39:12.080883 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:39:12.081195 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:39:12.081198 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:39:12.081200 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:39:12.081295 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:39:12.081821 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:39:12.081824 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:39:12.081937 (+ 113us) client_negotiation.cc:769] Sending connection context
0806 11:39:12.082056 (+ 119us) client_negotiation.cc:240] Negotiation successful
0806 11:39:12.082222 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"mutex_wait_us":29}
I20260806 11:39:12.082821 3523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:12.071559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60904 (local address 127.21.14.253:32951)
0806 11:39:12.071694 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:12.071698 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:12.071816 (+ 118us) server_negotiation.cc:407] Connection header received
0806 11:39:12.071949 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:12.071955 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:12.072004 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:12.072085 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:39:12.073048 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:12.073808 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:12.075087 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:12.075405 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:12.075524 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:12.078805 (+ 3281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:39:12.078809 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:39:12.078812 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:39:12.078837 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:39:12.080783 (+ 1946us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:12.081336 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:12.081342 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:12.081344 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:12.081441 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:12.081767 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:12.081772 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:12.081777 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:12.082019 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:39:12.082202 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:12.082532 (+ 330us) server_negotiation.cc:299] Negotiation successful
0806 11:39:12.082666 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 11:39:12.083141 3605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:12.071219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37078 (local address 127.21.14.252:34411)
0806 11:39:12.071328 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:12.071333 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:12.071404 (+ 71us) server_negotiation.cc:407] Connection header received
0806 11:39:12.071555 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:12.071558 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:12.071595 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:12.071666 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:39:12.072235 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:12.072801 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:12.073918 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:12.074315 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:12.074595 (+ 280us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:12.078541 (+ 3946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:39:12.078547 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:39:12.078554 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:39:12.078587 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:39:12.080528 (+ 1941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:12.080991 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:12.080995 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:12.080997 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:12.081095 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:12.081404 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:12.081407 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:12.081408 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:12.081696 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:39:12.081828 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:12.082777 (+ 949us) server_negotiation.cc:299] Negotiation successful
0806 11:39:12.082938 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":35}
I20260806 11:39:12.081248 3448 master_runner.cc:186] Error getting master registration for 127.21.14.254:33293: OK, instance_id { permanent_uuid: "4e36dac8bf0544e8bff9445954b5ba1b" instance_seqno: 1786016351542816 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
W20260806 11:39:12.085898 3621 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ce883cfa54124f00aaab713b62eeb554: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 11:39:12.086107 3621 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 11:39:12.085914 3448 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 11:39:12.089231 21563 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 11:39:12.089653 3600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87d10c23a1d0413a8c9d55caca6811ef: Bootstrap starting.
I20260806 11:39:12.093014 3600 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 87d10c23a1d0413a8c9d55caca6811ef: Neither blocks nor log segments found. Creating new log.
I20260806 11:39:12.094766 3600 log.cc:824] T 00000000000000000000000000000000 P 87d10c23a1d0413a8c9d55caca6811ef: Log is configured to *not* fsync() on all Append() calls
I20260806 11:39:12.098126 3600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87d10c23a1d0413a8c9d55caca6811ef: No bootstrap required, opened a new log
I20260806 11:39:12.104456 3600 raft_consensus.cc:348] T 00000000000000000000000000000000 P 87d10c23a1d0413a8c9d55caca6811ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e36dac8bf0544e8bff9445954b5ba1b" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 33293 } } peers { permanent_uuid: "ce883cfa54124f00aaab713b62eeb554" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 32951 } } peers { permanent_uuid: "87d10c23a1d0413a8c9d55caca6811ef" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 34411 } }
I20260806 11:39:12.104792 3600 raft_consensus.cc:374] T 00000000000000000000000000000000 P 87d10c23a1d0413a8c9d55caca6811ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:39:12.104856 3600 raft_consensus.cc:729] T 00000000000000000000000000000000 P 87d10c23a1d0413a8c9d55caca6811ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87d10c23a1d0413a8c9d55caca6811ef, State: Initialized, Role: FOLLOWER
I20260806 11:39:12.105479 3600 consensus_queue.cc:260] T 00000000000000000000000000000000 P 87d10c23a1d0413a8c9d55caca6811ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e36dac8bf0544e8bff9445954b5ba1b" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 33293 } } peers { permanent_uuid: "ce883cfa54124f00aaab713b62eeb554" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 32951 } } peers { permanent_uuid: "87d10c23a1d0413a8c9d55caca6811ef" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 34411 } }
I20260806 11:39:12.105921 3444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:12.092503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41116 (local address 127.21.14.254:33293)
0806 11:39:12.092652 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:12.092657 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:12.092913 (+ 256us) server_negotiation.cc:407] Connection header received
0806 11:39:12.093547 (+ 634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:12.093550 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:12.093602 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:12.093661 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:39:12.094907 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:12.095720 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:12.096640 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:12.097046 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:12.097195 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:12.099367 (+ 2172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:39:12.099372 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:39:12.099374 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:39:12.099409 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:39:12.101751 (+ 2342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:12.102683 (+ 932us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:12.102688 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:12.102691 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:12.103131 (+ 440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:12.103519 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:12.103523 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:12.103526 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:12.103952 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:39:12.104112 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:12.105364 (+ 1252us) server_negotiation.cc:299] Negotiation successful
0806 11:39:12.105544 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 11:39:12.106683 3523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:12.093334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60912 (local address 127.21.14.253:32951)
0806 11:39:12.093479 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:12.093482 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:12.095124 (+ 1642us) server_negotiation.cc:407] Connection header received
0806 11:39:12.095316 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:12.095319 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:12.095373 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:12.095424 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:39:12.097276 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:12.098081 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:12.099343 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:12.099712 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:12.099825 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:12.101828 (+ 2003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:39:12.101833 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:39:12.101835 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:39:12.101869 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:39:12.103964 (+ 2095us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:12.104599 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:12.104603 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:12.104605 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:12.104705 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:12.105300 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:12.105306 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:12.105308 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:12.105581 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:39:12.105723 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:12.106342 (+ 619us) server_negotiation.cc:299] Negotiation successful
0806 11:39:12.106509 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 11:39:12.107234 3600 sys_catalog.cc:564] T 00000000000000000000000000000000 P 87d10c23a1d0413a8c9d55caca6811ef [sys.catalog]: configured and running, proceeding with master startup.
I20260806 11:39:12.111281 3630 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87d10c23a1d0413a8c9d55caca6811ef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e36dac8bf0544e8bff9445954b5ba1b" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 33293 } } peers { permanent_uuid: "ce883cfa54124f00aaab713b62eeb554" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 32951 } } peers { permanent_uuid: "87d10c23a1d0413a8c9d55caca6811ef" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 34411 } } }
I20260806 11:39:12.112172 3630 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87d10c23a1d0413a8c9d55caca6811ef [sys.catalog]: This master's current role is: FOLLOWER
W20260806 11:39:12.114084 3642 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 87d10c23a1d0413a8c9d55caca6811ef: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 11:39:12.114266 3642 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 11:39:12.112564 3527 master_runner.cc:186] Error getting master registration for 127.21.14.254:33293: OK, instance_id { permanent_uuid: "4e36dac8bf0544e8bff9445954b5ba1b" instance_seqno: 1786016351542816 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 11:39:12.117188 3527 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 11:39:12.120724 3605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:12.108715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37086 (local address 127.21.14.252:34411)
0806 11:39:12.108871 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:12.108875 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:12.109296 (+ 421us) server_negotiation.cc:407] Connection header received
0806 11:39:12.109527 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:12.109530 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:12.109586 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:12.109641 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:39:12.110799 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:12.111596 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:12.113868 (+ 2272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:12.114227 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:12.114322 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:12.116390 (+ 2068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:39:12.116395 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:39:12.116398 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:39:12.116432 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:39:12.118389 (+ 1957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:12.119032 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:12.119038 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:12.119040 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:12.119146 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:12.119500 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:12.119504 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:12.119506 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:12.119781 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:39:12.119937 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:12.120361 (+ 424us) server_negotiation.cc:299] Negotiation successful
0806 11:39:12.120520 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 11:39:12.188992 3441 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.14.252" port: 34411 } attempt: 4
Aug 06 11:39:12 dist-test-slave-7c35 krb5kdc[3355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016351, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.254@KRBTEST.COM for kudu/127.21.14.252@KRBTEST.COM
I20260806 11:39:12.203270 3605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:12.189914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37096 (local address 127.21.14.252:34411)
0806 11:39:12.190056 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:12.190061 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:12.190561 (+ 500us) server_negotiation.cc:407] Connection header received
0806 11:39:12.190986 (+ 425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:12.190990 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:12.191051 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:12.191169 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:39:12.191798 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:12.192636 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:12.193855 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:12.194439 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:12.194581 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:12.197784 (+ 3203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:39:12.197790 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:39:12.197793 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:39:12.197833 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:39:12.200787 (+ 2954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:12.201484 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:12.201488 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:12.201490 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:12.201613 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:12.201999 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:12.202003 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:12.202004 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:12.202263 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:39:12.202370 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:12.202949 (+ 579us) server_negotiation.cc:299] Negotiation successful
0806 11:39:12.203089 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 11:39:12.203284 3445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:12.189969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.252:34411 (local address 127.0.0.1:37096)
0806 11:39:12.190409 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:12.190437 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 11:39:12.190767 (+ 330us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:39:12.191207 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:39:12.191211 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:39:12.191471 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:39:12.191619 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:12.191626 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:12.192796 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:12.192801 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:12.193690 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:12.193703 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:12.194640 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:12.194644 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:12.194752 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:12.195452 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:39:12.195470 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:39:12.197325 (+ 1855us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:39:12.200952 (+ 3627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:39:12.200956 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:39:12.200959 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:39:12.201367 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:39:12.201738 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:39:12.201742 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:39:12.201744 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:39:12.201883 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:39:12.202552 (+ 669us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:39:12.202562 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:39:12.202704 (+ 142us) client_negotiation.cc:769] Sending connection context
0806 11:39:12.202874 (+ 170us) client_negotiation.cc:240] Negotiation successful
0806 11:39:12.203073 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154}
I20260806 11:39:12.207866 3441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e36dac8bf0544e8bff9445954b5ba1b: Bootstrap starting.
I20260806 11:39:12.212831 3441 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4e36dac8bf0544e8bff9445954b5ba1b: Neither blocks nor log segments found. Creating new log.
I20260806 11:39:12.214112 3441 log.cc:824] T 00000000000000000000000000000000 P 4e36dac8bf0544e8bff9445954b5ba1b: Log is configured to *not* fsync() on all Append() calls
I20260806 11:39:12.217422 3441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e36dac8bf0544e8bff9445954b5ba1b: No bootstrap required, opened a new log
I20260806 11:39:12.222220 3441 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e36dac8bf0544e8bff9445954b5ba1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e36dac8bf0544e8bff9445954b5ba1b" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 33293 } } peers { permanent_uuid: "ce883cfa54124f00aaab713b62eeb554" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 32951 } } peers { permanent_uuid: "87d10c23a1d0413a8c9d55caca6811ef" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 34411 } }
I20260806 11:39:12.222589 3441 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4e36dac8bf0544e8bff9445954b5ba1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:39:12.222656 3441 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e36dac8bf0544e8bff9445954b5ba1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e36dac8bf0544e8bff9445954b5ba1b, State: Initialized, Role: FOLLOWER
I20260806 11:39:12.223593 3441 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e36dac8bf0544e8bff9445954b5ba1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e36dac8bf0544e8bff9445954b5ba1b" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 33293 } } peers { permanent_uuid: "ce883cfa54124f00aaab713b62eeb554" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 32951 } } peers { permanent_uuid: "87d10c23a1d0413a8c9d55caca6811ef" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 34411 } }
I20260806 11:39:12.224850 3645 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e36dac8bf0544e8bff9445954b5ba1b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e36dac8bf0544e8bff9445954b5ba1b" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 33293 } } peers { permanent_uuid: "ce883cfa54124f00aaab713b62eeb554" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 32951 } } peers { permanent_uuid: "87d10c23a1d0413a8c9d55caca6811ef" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 34411 } } }
I20260806 11:39:12.225201 3645 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e36dac8bf0544e8bff9445954b5ba1b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 11:39:12.226264 3441 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e36dac8bf0544e8bff9445954b5ba1b [sys.catalog]: configured and running, proceeding with master startup.
W20260806 11:39:12.233707 3656 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4e36dac8bf0544e8bff9445954b5ba1b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 11:39:12.234135 3656 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 11:39:12.239501 3368 master_runner.cc:263] No leader master found from master 4e36dac8bf0544e8bff9445954b5ba1b
I20260806 11:39:12.239621 3368 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 11:39:12.271982 3551 tablet.cc:2369] T 00000000000000000000000000000000 P 87d10c23a1d0413a8c9d55caca6811ef: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 11:39:12.274098 3471 tablet.cc:2369] T 00000000000000000000000000000000 P ce883cfa54124f00aaab713b62eeb554: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 11:39:12.289160 3389 tablet.cc:2369] T 00000000000000000000000000000000 P 4e36dac8bf0544e8bff9445954b5ba1b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 11:39:12.421486 3607 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce883cfa54124f00aaab713b62eeb554 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 11:39:12.421782 3607 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce883cfa54124f00aaab713b62eeb554 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e36dac8bf0544e8bff9445954b5ba1b" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 33293 } } peers { permanent_uuid: "ce883cfa54124f00aaab713b62eeb554" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 32951 } } peers { permanent_uuid: "87d10c23a1d0413a8c9d55caca6811ef" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 34411 } }
I20260806 11:39:12.423017 3607 leader_election.cc:288] T 00000000000000000000000000000000 P ce883cfa54124f00aaab713b62eeb554 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4e36dac8bf0544e8bff9445954b5ba1b (127.21.14.254:33293), 87d10c23a1d0413a8c9d55caca6811ef (127.21.14.252:34411)
I20260806 11:39:12.423707 3412 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ce883cfa54124f00aaab713b62eeb554" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e36dac8bf0544e8bff9445954b5ba1b" is_pre_election: true
I20260806 11:39:12.423862 3572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ce883cfa54124f00aaab713b62eeb554" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87d10c23a1d0413a8c9d55caca6811ef" is_pre_election: true
I20260806 11:39:12.424120 3412 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4e36dac8bf0544e8bff9445954b5ba1b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce883cfa54124f00aaab713b62eeb554 in term 0.
I20260806 11:39:12.424340 3572 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 87d10c23a1d0413a8c9d55caca6811ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce883cfa54124f00aaab713b62eeb554 in term 0.
I20260806 11:39:12.424731 3468 leader_election.cc:302] T 00000000000000000000000000000000 P ce883cfa54124f00aaab713b62eeb554 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e36dac8bf0544e8bff9445954b5ba1b, ce883cfa54124f00aaab713b62eeb554; no voters:
I20260806 11:39:12.425153 3607 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce883cfa54124f00aaab713b62eeb554 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 11:39:12.425277 3607 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce883cfa54124f00aaab713b62eeb554 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 11:39:12.425341 3607 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce883cfa54124f00aaab713b62eeb554 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:39:12.426926 3607 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce883cfa54124f00aaab713b62eeb554 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e36dac8bf0544e8bff9445954b5ba1b" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 33293 } } peers { permanent_uuid: "ce883cfa54124f00aaab713b62eeb554" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 32951 } } peers { permanent_uuid: "87d10c23a1d0413a8c9d55caca6811ef" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 34411 } }
I20260806 11:39:12.427498 3607 leader_election.cc:288] T 00000000000000000000000000000000 P ce883cfa54124f00aaab713b62eeb554 [CANDIDATE]: Term 1 election: Requested vote from peers 4e36dac8bf0544e8bff9445954b5ba1b (127.21.14.254:33293), 87d10c23a1d0413a8c9d55caca6811ef (127.21.14.252:34411)
I20260806 11:39:12.427788 3412 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ce883cfa54124f00aaab713b62eeb554" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e36dac8bf0544e8bff9445954b5ba1b"
I20260806 11:39:12.427958 3412 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e36dac8bf0544e8bff9445954b5ba1b [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:39:12.428205 3572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ce883cfa54124f00aaab713b62eeb554" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87d10c23a1d0413a8c9d55caca6811ef"
I20260806 11:39:12.428334 3572 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 87d10c23a1d0413a8c9d55caca6811ef [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:39:12.429388 3412 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4e36dac8bf0544e8bff9445954b5ba1b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce883cfa54124f00aaab713b62eeb554 in term 1.
I20260806 11:39:12.429665 3572 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 87d10c23a1d0413a8c9d55caca6811ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce883cfa54124f00aaab713b62eeb554 in term 1.
I20260806 11:39:12.429750 3468 leader_election.cc:302] T 00000000000000000000000000000000 P ce883cfa54124f00aaab713b62eeb554 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e36dac8bf0544e8bff9445954b5ba1b, ce883cfa54124f00aaab713b62eeb554; no voters:
I20260806 11:39:12.429997 3607 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce883cfa54124f00aaab713b62eeb554 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:39:12.430292 3607 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce883cfa54124f00aaab713b62eeb554 [term 1 LEADER]: Becoming Leader. State: Replica: ce883cfa54124f00aaab713b62eeb554, State: Running, Role: LEADER
I20260806 11:39:12.430685 3607 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce883cfa54124f00aaab713b62eeb554 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e36dac8bf0544e8bff9445954b5ba1b" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 33293 } } peers { permanent_uuid: "ce883cfa54124f00aaab713b62eeb554" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 32951 } } peers { permanent_uuid: "87d10c23a1d0413a8c9d55caca6811ef" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 34411 } }
I20260806 11:39:12.432778 3658 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce883cfa54124f00aaab713b62eeb554 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce883cfa54124f00aaab713b62eeb554. Latest consensus state: current_term: 1 leader_uuid: "ce883cfa54124f00aaab713b62eeb554" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e36dac8bf0544e8bff9445954b5ba1b" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 33293 } } peers { permanent_uuid: "ce883cfa54124f00aaab713b62eeb554" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 32951 } } peers { permanent_uuid: "87d10c23a1d0413a8c9d55caca6811ef" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 34411 } } }
I20260806 11:39:12.432912 3658 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce883cfa54124f00aaab713b62eeb554 [sys.catalog]: This master's current role is: LEADER
I20260806 11:39:12.433300 3660 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 11:39:12.434211 3660 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 11:39:12.439778 3412 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4e36dac8bf0544e8bff9445954b5ba1b [term 1 FOLLOWER]: Refusing update from remote peer ce883cfa54124f00aaab713b62eeb554: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 11:39:12.439821 3572 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 87d10c23a1d0413a8c9d55caca6811ef [term 1 FOLLOWER]: Refusing update from remote peer ce883cfa54124f00aaab713b62eeb554: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 11:39:12.440519 3607 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ce883cfa54124f00aaab713b62eeb554 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e36dac8bf0544e8bff9445954b5ba1b" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 33293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 11:39:12.440744 3658 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ce883cfa54124f00aaab713b62eeb554 [LEADER]: Connected to new peer: Peer: permanent_uuid: "87d10c23a1d0413a8c9d55caca6811ef" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 34411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 11:39:12.445534 3630 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87d10c23a1d0413a8c9d55caca6811ef [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce883cfa54124f00aaab713b62eeb554. Latest consensus state: current_term: 1 leader_uuid: "ce883cfa54124f00aaab713b62eeb554" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e36dac8bf0544e8bff9445954b5ba1b" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 33293 } } peers { permanent_uuid: "ce883cfa54124f00aaab713b62eeb554" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 32951 } } peers { permanent_uuid: "87d10c23a1d0413a8c9d55caca6811ef" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 34411 } } }
I20260806 11:39:12.445583 3645 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e36dac8bf0544e8bff9445954b5ba1b [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce883cfa54124f00aaab713b62eeb554. Latest consensus state: current_term: 1 leader_uuid: "ce883cfa54124f00aaab713b62eeb554" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e36dac8bf0544e8bff9445954b5ba1b" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 33293 } } peers { permanent_uuid: "ce883cfa54124f00aaab713b62eeb554" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 32951 } } peers { permanent_uuid: "87d10c23a1d0413a8c9d55caca6811ef" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 34411 } } }
I20260806 11:39:12.445727 3645 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e36dac8bf0544e8bff9445954b5ba1b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 11:39:12.445724 3630 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87d10c23a1d0413a8c9d55caca6811ef [sys.catalog]: This master's current role is: FOLLOWER
I20260806 11:39:12.448362 3662 mvcc.cc:205] Tried to move back new op lower bound from 7315522979585224704 to 7315522979559280640. Current Snapshot: MvccSnapshot[applied={T|T < 7315522979585224704}]
I20260806 11:39:12.448557 3607 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce883cfa54124f00aaab713b62eeb554 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ce883cfa54124f00aaab713b62eeb554" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e36dac8bf0544e8bff9445954b5ba1b" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 33293 } } peers { permanent_uuid: "ce883cfa54124f00aaab713b62eeb554" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 32951 } } peers { permanent_uuid: "87d10c23a1d0413a8c9d55caca6811ef" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 34411 } } }
I20260806 11:39:12.448691 3607 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce883cfa54124f00aaab713b62eeb554 [sys.catalog]: This master's current role is: LEADER
I20260806 11:39:12.448797 3607 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce883cfa54124f00aaab713b62eeb554 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ce883cfa54124f00aaab713b62eeb554" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e36dac8bf0544e8bff9445954b5ba1b" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 33293 } } peers { permanent_uuid: "ce883cfa54124f00aaab713b62eeb554" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 32951 } } peers { permanent_uuid: "87d10c23a1d0413a8c9d55caca6811ef" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 34411 } } }
I20260806 11:39:12.448866 3607 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce883cfa54124f00aaab713b62eeb554 [sys.catalog]: This master's current role is: LEADER
I20260806 11:39:12.449146 3645 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e36dac8bf0544e8bff9445954b5ba1b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ce883cfa54124f00aaab713b62eeb554" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e36dac8bf0544e8bff9445954b5ba1b" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 33293 } } peers { permanent_uuid: "ce883cfa54124f00aaab713b62eeb554" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 32951 } } peers { permanent_uuid: "87d10c23a1d0413a8c9d55caca6811ef" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 34411 } } }
I20260806 11:39:12.449307 3645 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e36dac8bf0544e8bff9445954b5ba1b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 11:39:12.449534 3664 mvcc.cc:205] Tried to move back new op lower bound from 7315522979585224704 to 7315522979559280640. Current Snapshot: MvccSnapshot[applied={T|T < 7315522979585224704}]
I20260806 11:39:12.450033 3660 catalog_manager.cc:1347] Generated new cluster ID: 4dd9b5791ae34ab4b555081e68398353
I20260806 11:39:12.450112 3660 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 11:39:12.450510 3630 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87d10c23a1d0413a8c9d55caca6811ef [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ce883cfa54124f00aaab713b62eeb554" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e36dac8bf0544e8bff9445954b5ba1b" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 33293 } } peers { permanent_uuid: "ce883cfa54124f00aaab713b62eeb554" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 32951 } } peers { permanent_uuid: "87d10c23a1d0413a8c9d55caca6811ef" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 34411 } } }
I20260806 11:39:12.450651 3630 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87d10c23a1d0413a8c9d55caca6811ef [sys.catalog]: This master's current role is: FOLLOWER
I20260806 11:39:12.460434 3660 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 11:39:12.461669 3660 catalog_manager.cc:1504] Loading token signing keys...
I20260806 11:39:12.475340 3660 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ce883cfa54124f00aaab713b62eeb554: Generated new TSK 0
I20260806 11:39:12.476509 3660 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 11:39:13.116153 3642 catalog_manager.cc:1259] Loaded cluster ID: 4dd9b5791ae34ab4b555081e68398353
I20260806 11:39:13.116381 3642 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 87d10c23a1d0413a8c9d55caca6811ef: loading cluster ID for follower catalog manager: success
I20260806 11:39:13.118268 3642 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 87d10c23a1d0413a8c9d55caca6811ef: acquiring CA information for follower catalog manager: success
I20260806 11:39:13.119810 3642 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 87d10c23a1d0413a8c9d55caca6811ef: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 11:39:13.235749 3656 catalog_manager.cc:1259] Loaded cluster ID: 4dd9b5791ae34ab4b555081e68398353
I20260806 11:39:13.235850 3656 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4e36dac8bf0544e8bff9445954b5ba1b: loading cluster ID for follower catalog manager: success
I20260806 11:39:13.237661 3656 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4e36dac8bf0544e8bff9445954b5ba1b: acquiring CA information for follower catalog manager: success
I20260806 11:39:13.239038 3656 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4e36dac8bf0544e8bff9445954b5ba1b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 11:39:13.542701 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 3368
I20260806 11:39:13.552176 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 3448
I20260806 11:39:13.562012 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 3527
2026-08-06T11:39:13Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3501 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786016169045816-21563-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 11:39:13 dist-test-slave-7c35 krb5kdc[3676](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 11:39:13 dist-test-slave-7c35 krb5kdc[3676](info): set up 2 sockets
Aug 06 11:39:13 dist-test-slave-7c35 krb5kdc[3676](info): commencing operation
krb5kdc: starting...
W20260806 11:39:14.675330 21563 mini_kdc.cc:121] Time spent starting KDC: real 1.088s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Aug 06 11:39:14 dist-test-slave-7c35 krb5kdc[3676](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016354, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T11:39:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T11:39:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.14.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab.
Entry for principal kudu/127.21.14.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab.
I20260806 11:39:14.822965 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786016169045816-21563-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786016169045816-21563-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786016169045816-21563-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:41997
--webserver_interface=127.21.14.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab
--principal=kudu/127.21.14.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:34865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.14.254:41997
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:39:14.943813 3690 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:39:14.944098 3690 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:39:14.944140 3690 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:39:14.948339 3690 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 11:39:14.948414 3690 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:39:14.948434 3690 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:39:14.948451 3690 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 11:39:14.948467 3690 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 11:39:14.954072 3690 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:34865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786016169045816-21563-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786016169045816-21563-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.14.254:41997
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab
--principal=kudu/127.21.14.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:41997
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.14.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3690
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786016169045816-21563-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:39:14.955358 3690 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:39:14.956450 3690 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:39:14.965664 3700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:39:14.965664 3697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:39:14.965687 3698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:39:14.966668 3690 server_base.cc:1034] running on GCE node
I20260806 11:39:14.967330 3690 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:39:14.968549 3690 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:39:14.969766 3690 hybrid_clock.cc:648] HybridClock initialized: now 1786016354969732 us; error 53 us; skew 500 ppm
Aug 06 11:39:14 dist-test-slave-7c35 krb5kdc[3676](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016354, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:39:14.972908 3690 init.cc:377] Logged in from keytab as kudu/127.21.14.254@KRBTEST.COM (short username kudu)
I20260806 11:39:14.973939 3690 webserver.cc:460] Webserver started at http://127.21.14.254:38941/ using document root <none> and password file <none>
I20260806 11:39:14.974583 3690 fs_manager.cc:359] Metadata directory not provided
I20260806 11:39:14.974684 3690 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:39:14.974996 3690 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:39:14.977240 3690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786016169045816-21563-0/minicluster-data/master-0/data/instance:
uuid: "670d0250764347ee8f09c6eab9c0a13c"
format_stamp: "Formatted at 2026-08-06 11:39:14 on dist-test-slave-7c35"
server_key: "7212469f9d5b2dbc2ac0e6363b61b363"
server_key_iv: "8846b60b2975818a7286ffe02a453456"
server_key_version: "encryptionkey@0"
I20260806 11:39:14.977855 3690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786016169045816-21563-0/minicluster-data/master-0/wal/instance:
uuid: "670d0250764347ee8f09c6eab9c0a13c"
format_stamp: "Formatted at 2026-08-06 11:39:14 on dist-test-slave-7c35"
server_key: "7212469f9d5b2dbc2ac0e6363b61b363"
server_key_iv: "8846b60b2975818a7286ffe02a453456"
server_key_version: "encryptionkey@0"
I20260806 11:39:14.982124 3690 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 11:39:14.985023 3708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:39:14.986284 3690 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 11:39:14.986444 3690 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786016169045816-21563-0/minicluster-data/master-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786016169045816-21563-0/minicluster-data/master-0/wal
uuid: "670d0250764347ee8f09c6eab9c0a13c"
format_stamp: "Formatted at 2026-08-06 11:39:14 on dist-test-slave-7c35"
server_key: "7212469f9d5b2dbc2ac0e6363b61b363"
server_key_iv: "8846b60b2975818a7286ffe02a453456"
server_key_version: "encryptionkey@0"
I20260806 11:39:14.986588 3690 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786016169045816-21563-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786016169045816-21563-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786016169045816-21563-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:39:15.000742 3690 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:39:15.004060 3690 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:39:15.004240 3690 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:39:15.011795 3690 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.254:41997
I20260806 11:39:15.011823 3762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.254:41997 every 8 connection(s)
I20260806 11:39:15.012873 3690 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
I20260806 11:39:15.015913 3763 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:39:15.021438 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 3690
I20260806 11:39:15.021566 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786016169045816-21563-0/minicluster-data/master-0/wal/instance
I20260806 11:39:15.021857 21563 external_mini_cluster.cc:1428] Setting key 58386cb5b771079600eacc1c114b9949
I20260806 11:39:15.021883 3763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 670d0250764347ee8f09c6eab9c0a13c: Bootstrap starting.
I20260806 11:39:15.024642 3763 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 670d0250764347ee8f09c6eab9c0a13c: Neither blocks nor log segments found. Creating new log.
I20260806 11:39:15.025456 3763 log.cc:824] T 00000000000000000000000000000000 P 670d0250764347ee8f09c6eab9c0a13c: Log is configured to *not* fsync() on all Append() calls
I20260806 11:39:15.027949 3763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 670d0250764347ee8f09c6eab9c0a13c: No bootstrap required, opened a new log
Aug 06 11:39:15 dist-test-slave-7c35 krb5kdc[3676](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016354, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:39:15.032853 3763 raft_consensus.cc:348] T 00000000000000000000000000000000 P 670d0250764347ee8f09c6eab9c0a13c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "670d0250764347ee8f09c6eab9c0a13c" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 41997 } }
I20260806 11:39:15.033201 3763 raft_consensus.cc:374] T 00000000000000000000000000000000 P 670d0250764347ee8f09c6eab9c0a13c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:39:15.033286 3763 raft_consensus.cc:729] T 00000000000000000000000000000000 P 670d0250764347ee8f09c6eab9c0a13c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 670d0250764347ee8f09c6eab9c0a13c, State: Initialized, Role: FOLLOWER
I20260806 11:39:15.033881 3763 consensus_queue.cc:260] T 00000000000000000000000000000000 P 670d0250764347ee8f09c6eab9c0a13c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "670d0250764347ee8f09c6eab9c0a13c" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 41997 } }
I20260806 11:39:15.034063 3763 raft_consensus.cc:388] T 00000000000000000000000000000000 P 670d0250764347ee8f09c6eab9c0a13c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 11:39:15.034147 3763 raft_consensus.cc:482] T 00000000000000000000000000000000 P 670d0250764347ee8f09c6eab9c0a13c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 11:39:15.034260 3763 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 670d0250764347ee8f09c6eab9c0a13c [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:39:15.035403 3763 raft_consensus.cc:504] T 00000000000000000000000000000000 P 670d0250764347ee8f09c6eab9c0a13c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "670d0250764347ee8f09c6eab9c0a13c" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 41997 } }
I20260806 11:39:15.035807 3763 leader_election.cc:302] T 00000000000000000000000000000000 P 670d0250764347ee8f09c6eab9c0a13c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 670d0250764347ee8f09c6eab9c0a13c; no voters:
I20260806 11:39:15.036207 3763 leader_election.cc:288] T 00000000000000000000000000000000 P 670d0250764347ee8f09c6eab9c0a13c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 11:39:15.036306 3768 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 670d0250764347ee8f09c6eab9c0a13c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:39:15.036557 3768 raft_consensus.cc:686] T 00000000000000000000000000000000 P 670d0250764347ee8f09c6eab9c0a13c [term 1 LEADER]: Becoming Leader. State: Replica: 670d0250764347ee8f09c6eab9c0a13c, State: Running, Role: LEADER
I20260806 11:39:15.036947 3768 consensus_queue.cc:237] T 00000000000000000000000000000000 P 670d0250764347ee8f09c6eab9c0a13c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "670d0250764347ee8f09c6eab9c0a13c" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 41997 } }
I20260806 11:39:15.037711 3763 sys_catalog.cc:564] T 00000000000000000000000000000000 P 670d0250764347ee8f09c6eab9c0a13c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 11:39:15.038285 3768 sys_catalog.cc:455] T 00000000000000000000000000000000 P 670d0250764347ee8f09c6eab9c0a13c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 670d0250764347ee8f09c6eab9c0a13c. Latest consensus state: current_term: 1 leader_uuid: "670d0250764347ee8f09c6eab9c0a13c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "670d0250764347ee8f09c6eab9c0a13c" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 41997 } } }
I20260806 11:39:15.038497 3768 sys_catalog.cc:458] T 00000000000000000000000000000000 P 670d0250764347ee8f09c6eab9c0a13c [sys.catalog]: This master's current role is: LEADER
I20260806 11:39:15.038717 3769 sys_catalog.cc:455] T 00000000000000000000000000000000 P 670d0250764347ee8f09c6eab9c0a13c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "670d0250764347ee8f09c6eab9c0a13c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "670d0250764347ee8f09c6eab9c0a13c" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 41997 } } }
I20260806 11:39:15.038918 3769 sys_catalog.cc:458] T 00000000000000000000000000000000 P 670d0250764347ee8f09c6eab9c0a13c [sys.catalog]: This master's current role is: LEADER
I20260806 11:39:15.039491 3766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:15.023308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37052 (local address 127.21.14.254:41997)
0806 11:39:15.023817 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:15.023833 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:15.023871 (+ 38us) server_negotiation.cc:407] Connection header received
0806 11:39:15.024539 (+ 668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:15.024559 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:15.024981 (+ 422us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:15.025346 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:39:15.026346 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:15.027875 (+ 1529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:15.028820 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:15.029534 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:15.029713 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:15.032718 (+ 3005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:39:15.032725 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:39:15.032747 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:39:15.032783 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:39:15.035109 (+ 2326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:15.035695 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:15.035700 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:15.035706 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:15.035846 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:15.036184 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:15.036191 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:15.036193 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:15.036847 (+ 654us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:39:15.037729 (+ 882us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:15.037894 (+ 165us) server_negotiation.cc:299] Negotiation successful
0806 11:39:15.038227 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":148,"threads_started":1}
I20260806 11:39:15.042830 21563 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260806 11:39:15.043854 3784 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 670d0250764347ee8f09c6eab9c0a13c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 11:39:15.043958 3784 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 11:39:15.044034 3775 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 11:39:15.044931 3775 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 11:39:15.051134 3775 catalog_manager.cc:1347] Generated new cluster ID: e00f30a5477c442caf9661cad30a7cca
I20260806 11:39:15.051225 3775 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 11:39:15.067610 3775 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 11:39:15.068619 3775 catalog_manager.cc:1504] Loading token signing keys...
I20260806 11:39:15.075371 3775 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 670d0250764347ee8f09c6eab9c0a13c: Generated new TSK 0
I20260806 11:39:15.076212 3775 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 11:39:15.100284 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 3690
2026-08-06T11:39:15Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1538 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 11:39:15 dist-test-slave-7c35 krb5kdc[3793](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 11:39:15 dist-test-slave-7c35 krb5kdc[3793](info): set up 2 sockets
Aug 06 11:39:15 dist-test-slave-7c35 krb5kdc[3793](info): commencing operation
krb5kdc: starting...
W20260806 11:39:16.208138 21563 mini_kdc.cc:121] Time spent starting KDC: real 1.082s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Aug 06 11:39:16 dist-test-slave-7c35 krb5kdc[3793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016356, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T11:39:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T11:39:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.14.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab.
Entry for principal kudu/127.21.14.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab.
I20260806 11:39:16.355822 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:37871
--webserver_interface=127.21.14.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab
--principal=kudu/127.21.14.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:45535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.14.254:37871
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:39:16.478629 3806 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:39:16.479351 3806 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:39:16.479489 3806 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:39:16.483963 3806 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 11:39:16.484057 3806 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:39:16.484095 3806 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:39:16.484136 3806 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 11:39:16.484171 3806 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 11:39:16.490188 3806 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:45535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.14.254:37871
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab
--principal=kudu/127.21.14.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:37871
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.14.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3806
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:39:16.491484 3806 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:39:16.492604 3806 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:39:16.501678 3814 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:39:16.501740 3806 server_base.cc:1034] running on GCE node
W20260806 11:39:16.501699 3811 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:39:16.501682 3812 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:39:16.502568 3806 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:39:16.503830 3806 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:39:16.505002 3806 hybrid_clock.cc:648] HybridClock initialized: now 1786016356504988 us; error 30 us; skew 500 ppm
Aug 06 11:39:16 dist-test-slave-7c35 krb5kdc[3793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016356, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:39:16.507992 3806 init.cc:377] Logged in from keytab as kudu/127.21.14.254@KRBTEST.COM (short username kudu)
I20260806 11:39:16.508919 3806 webserver.cc:460] Webserver started at http://127.21.14.254:38567/ using document root <none> and password file <none>
I20260806 11:39:16.509488 3806 fs_manager.cc:359] Metadata directory not provided
I20260806 11:39:16.509547 3806 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:39:16.509727 3806 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:39:16.511970 3806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/master-0/data/instance:
uuid: "beea7743cac049cfb113cc825011d29e"
format_stamp: "Formatted at 2026-08-06 11:39:16 on dist-test-slave-7c35"
server_key: "c8ab8bf50e62fe9626dfaa8e07ca149d"
server_key_iv: "05fa07ec7bb16e4ef5d888862e737b13"
server_key_version: "encryptionkey@0"
I20260806 11:39:16.512606 3806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/master-0/wal/instance:
uuid: "beea7743cac049cfb113cc825011d29e"
format_stamp: "Formatted at 2026-08-06 11:39:16 on dist-test-slave-7c35"
server_key: "c8ab8bf50e62fe9626dfaa8e07ca149d"
server_key_iv: "05fa07ec7bb16e4ef5d888862e737b13"
server_key_version: "encryptionkey@0"
I20260806 11:39:16.516722 3806 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 11:39:16.519603 3821 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:39:16.520943 3806 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 11:39:16.521113 3806 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/master-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/master-0/wal
uuid: "beea7743cac049cfb113cc825011d29e"
format_stamp: "Formatted at 2026-08-06 11:39:16 on dist-test-slave-7c35"
server_key: "c8ab8bf50e62fe9626dfaa8e07ca149d"
server_key_iv: "05fa07ec7bb16e4ef5d888862e737b13"
server_key_version: "encryptionkey@0"
I20260806 11:39:16.521245 3806 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:39:16.536985 3806 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:39:16.540426 3806 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:39:16.540659 3806 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:39:16.548295 3806 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.254:37871
I20260806 11:39:16.548658 3877 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.254:37871 every 8 connection(s)
I20260806 11:39:16.549394 3806 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
I20260806 11:39:16.552485 3878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:39:16.554701 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 3806
I20260806 11:39:16.554884 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/master-0/wal/instance
I20260806 11:39:16.555190 21563 external_mini_cluster.cc:1428] Setting key e281a1df2448d4bc0cf580a42de03eb7
I20260806 11:39:16.560256 3878 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P beea7743cac049cfb113cc825011d29e: Bootstrap starting.
I20260806 11:39:16.563208 3878 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P beea7743cac049cfb113cc825011d29e: Neither blocks nor log segments found. Creating new log.
I20260806 11:39:16.564133 3878 log.cc:824] T 00000000000000000000000000000000 P beea7743cac049cfb113cc825011d29e: Log is configured to *not* fsync() on all Append() calls
Aug 06 11:39:16 dist-test-slave-7c35 krb5kdc[3793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016356, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:39:16.566713 3878 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P beea7743cac049cfb113cc825011d29e: No bootstrap required, opened a new log
I20260806 11:39:16.571753 3882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:16.556760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47746 (local address 127.21.14.254:37871)
0806 11:39:16.557244 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:16.557272 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:16.557318 (+ 46us) server_negotiation.cc:407] Connection header received
0806 11:39:16.558122 (+ 804us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:16.558151 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:16.558619 (+ 468us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:16.559078 (+ 459us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:39:16.559754 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:16.561256 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:16.562173 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:16.562903 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:16.563106 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:16.565972 (+ 2866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:39:16.565979 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:39:16.565997 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:39:16.566029 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:39:16.568179 (+ 2150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:16.568690 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:16.568697 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:16.568704 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:16.568853 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:16.569136 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:16.569139 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:16.569141 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:16.569776 (+ 635us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:39:16.569968 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:16.570282 (+ 314us) server_negotiation.cc:299] Negotiation successful
0806 11:39:16.570638 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":134,"threads_started":1}
I20260806 11:39:16.571767 3878 raft_consensus.cc:348] T 00000000000000000000000000000000 P beea7743cac049cfb113cc825011d29e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beea7743cac049cfb113cc825011d29e" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 37871 } }
I20260806 11:39:16.572122 3878 raft_consensus.cc:374] T 00000000000000000000000000000000 P beea7743cac049cfb113cc825011d29e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:39:16.572184 3878 raft_consensus.cc:729] T 00000000000000000000000000000000 P beea7743cac049cfb113cc825011d29e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: beea7743cac049cfb113cc825011d29e, State: Initialized, Role: FOLLOWER
I20260806 11:39:16.572641 3878 consensus_queue.cc:260] T 00000000000000000000000000000000 P beea7743cac049cfb113cc825011d29e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beea7743cac049cfb113cc825011d29e" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 37871 } }
I20260806 11:39:16.572767 3878 raft_consensus.cc:388] T 00000000000000000000000000000000 P beea7743cac049cfb113cc825011d29e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 11:39:16.572856 3878 raft_consensus.cc:482] T 00000000000000000000000000000000 P beea7743cac049cfb113cc825011d29e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 11:39:16.572957 3878 raft_consensus.cc:3037] T 00000000000000000000000000000000 P beea7743cac049cfb113cc825011d29e [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:39:16.574313 3878 raft_consensus.cc:504] T 00000000000000000000000000000000 P beea7743cac049cfb113cc825011d29e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beea7743cac049cfb113cc825011d29e" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 37871 } }
I20260806 11:39:16.574702 3878 leader_election.cc:302] T 00000000000000000000000000000000 P beea7743cac049cfb113cc825011d29e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: beea7743cac049cfb113cc825011d29e; no voters:
I20260806 11:39:16.575055 3878 leader_election.cc:288] T 00000000000000000000000000000000 P beea7743cac049cfb113cc825011d29e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 11:39:16.575186 3884 raft_consensus.cc:2781] T 00000000000000000000000000000000 P beea7743cac049cfb113cc825011d29e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:39:16.575445 3884 raft_consensus.cc:686] T 00000000000000000000000000000000 P beea7743cac049cfb113cc825011d29e [term 1 LEADER]: Becoming Leader. State: Replica: beea7743cac049cfb113cc825011d29e, State: Running, Role: LEADER
I20260806 11:39:16.575876 3884 consensus_queue.cc:237] T 00000000000000000000000000000000 P beea7743cac049cfb113cc825011d29e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beea7743cac049cfb113cc825011d29e" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 37871 } }
I20260806 11:39:16.576392 3878 sys_catalog.cc:564] T 00000000000000000000000000000000 P beea7743cac049cfb113cc825011d29e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 11:39:16.577625 3887 sys_catalog.cc:455] T 00000000000000000000000000000000 P beea7743cac049cfb113cc825011d29e [sys.catalog]: SysCatalogTable state changed. Reason: New leader beea7743cac049cfb113cc825011d29e. Latest consensus state: current_term: 1 leader_uuid: "beea7743cac049cfb113cc825011d29e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beea7743cac049cfb113cc825011d29e" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 37871 } } }
I20260806 11:39:16.577773 3887 sys_catalog.cc:458] T 00000000000000000000000000000000 P beea7743cac049cfb113cc825011d29e [sys.catalog]: This master's current role is: LEADER
I20260806 11:39:16.578042 3886 sys_catalog.cc:455] T 00000000000000000000000000000000 P beea7743cac049cfb113cc825011d29e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "beea7743cac049cfb113cc825011d29e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beea7743cac049cfb113cc825011d29e" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 37871 } } }
I20260806 11:39:16.578142 3886 sys_catalog.cc:458] T 00000000000000000000000000000000 P beea7743cac049cfb113cc825011d29e [sys.catalog]: This master's current role is: LEADER
W20260806 11:39:16.581653 3900 catalog_manager.cc:1558] T 00000000000000000000000000000000 P beea7743cac049cfb113cc825011d29e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 11:39:16.581734 3900 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 11:39:16.581812 3894 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 11:39:16.582590 3894 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 11:39:16.588861 3894 catalog_manager.cc:1347] Generated new cluster ID: 29f8b5ac6b7845f9a0cc9327663aa7ac
I20260806 11:39:16.588974 3894 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 11:39:16.603286 3894 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 11:39:16.604362 3894 catalog_manager.cc:1504] Loading token signing keys...
I20260806 11:39:16.614893 3894 catalog_manager.cc:5868] T 00000000000000000000000000000000 P beea7743cac049cfb113cc825011d29e: Generated new TSK 0
I20260806 11:39:16.615769 3894 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.14.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab.
Entry for principal kudu/127.21.14.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab.
I20260806 11:39:16.650590 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.193:0
--local_ip_for_outbound_sockets=127.21.14.193
--webserver_interface=127.21.14.193
--webserver_port=0
--tserver_master_addrs=127.21.14.254:37871
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab
--principal=kudu/127.21.14.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:45535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:39:16.774098 3906 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:39:16.774412 3906 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:39:16.774490 3906 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:39:16.779031 3906 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:39:16.779119 3906 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:39:16.779227 3906 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.193
I20260806 11:39:16.785300 3906 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:45535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab
--principal=kudu/127.21.14.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.14.193
--webserver_port=0
--tserver_master_addrs=127.21.14.254:37871
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3906
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.193
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:39:16.786553 3906 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:39:16.787734 3906 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:39:16.798504 3912 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:39:16.798595 3914 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:39:16.798715 3906 server_base.cc:1034] running on GCE node
W20260806 11:39:16.798504 3911 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:39:16.799398 3906 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:39:16.800159 3906 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:39:16.801383 3906 hybrid_clock.cc:648] HybridClock initialized: now 1786016356801327 us; error 69 us; skew 500 ppm
Aug 06 11:39:16 dist-test-slave-7c35 krb5kdc[3793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016356, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:39:16.804579 3906 init.cc:377] Logged in from keytab as kudu/127.21.14.193@KRBTEST.COM (short username kudu)
I20260806 11:39:16.805541 3906 webserver.cc:460] Webserver started at http://127.21.14.193:35725/ using document root <none> and password file <none>
I20260806 11:39:16.806119 3906 fs_manager.cc:359] Metadata directory not provided
I20260806 11:39:16.806183 3906 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:39:16.806362 3906 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:39:16.808672 3906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-0/data/instance:
uuid: "6ce2bc1867f74eb6b113467463a9430a"
format_stamp: "Formatted at 2026-08-06 11:39:16 on dist-test-slave-7c35"
server_key: "26832165400930665ce02c708c257e99"
server_key_iv: "8493c9f433424ba15d78243c5433e296"
server_key_version: "encryptionkey@0"
I20260806 11:39:16.809233 3906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-0/wal/instance:
uuid: "6ce2bc1867f74eb6b113467463a9430a"
format_stamp: "Formatted at 2026-08-06 11:39:16 on dist-test-slave-7c35"
server_key: "26832165400930665ce02c708c257e99"
server_key_iv: "8493c9f433424ba15d78243c5433e296"
server_key_version: "encryptionkey@0"
I20260806 11:39:16.813285 3906 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 11:39:16.815884 3921 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:39:16.817355 3906 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 11:39:16.817485 3906 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-0/wal
uuid: "6ce2bc1867f74eb6b113467463a9430a"
format_stamp: "Formatted at 2026-08-06 11:39:16 on dist-test-slave-7c35"
server_key: "26832165400930665ce02c708c257e99"
server_key_iv: "8493c9f433424ba15d78243c5433e296"
server_key_version: "encryptionkey@0"
I20260806 11:39:16.817602 3906 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:39:16.851428 3906 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:39:16.854547 3906 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:39:16.854709 3906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:39:16.855300 3906 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:39:16.856146 3906 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:39:16.856196 3906 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:39:16.856238 3906 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:39:16.856253 3906 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:39:16.865644 3906 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.193:43653
I20260806 11:39:16.865671 4043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.193:43653 every 8 connection(s)
I20260806 11:39:16.866606 3906 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
I20260806 11:39:16.869269 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 3906
I20260806 11:39:16.869426 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-0/wal/instance
I20260806 11:39:16.869769 21563 external_mini_cluster.cc:1428] Setting key 0ca90b4f6a231a4c76ca065aa60f54b3
Aug 06 11:39:16 dist-test-slave-7c35 krb5kdc[3793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016356, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.193@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.14.194@KRBTEST.COM; defaulting to no policy
I20260806 11:39:16.884014 3882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:16.868624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:38105 (local address 127.21.14.254:37871)
0806 11:39:16.868778 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:16.868782 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:16.869698 (+ 916us) server_negotiation.cc:407] Connection header received
0806 11:39:16.871553 (+ 1855us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:16.871556 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:16.871611 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:16.871701 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:39:16.872960 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:16.873795 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:16.875739 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:16.875986 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:16.876072 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:16.879739 (+ 3667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:39:16.879743 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:39:16.879745 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:39:16.879772 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:39:16.881326 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:16.882082 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:16.882086 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:16.882087 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:16.882170 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:16.882625 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:16.882628 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:16.882629 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:16.882973 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:39:16.883071 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:16.883675 (+ 604us) server_negotiation.cc:299] Negotiation successful
0806 11:39:16.883808 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.194@KRBTEST.COM" created.
I20260806 11:39:16.884938 4046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:16.868990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:37871 (local address 127.21.14.193:38105)
0806 11:39:16.869518 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:16.869578 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 11:39:16.871282 (+ 1704us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:39:16.871869 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:39:16.871878 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:39:16.872268 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:39:16.872804 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:16.872818 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:16.873937 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:16.873943 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:16.875599 (+ 1656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:16.875614 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:16.876156 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:16.876163 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:16.876425 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:16.877245 (+ 820us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:39:16.877275 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:39:16.879520 (+ 2245us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:39:16.881467 (+ 1947us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:39:16.881475 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:39:16.881495 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:39:16.881922 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:39:16.882301 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:39:16.882304 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:39:16.882305 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:39:16.882480 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:39:16.883094 (+ 614us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:39:16.883100 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:39:16.883388 (+ 288us) client_negotiation.cc:769] Sending connection context
0806 11:39:16.883566 (+ 178us) client_negotiation.cc:240] Negotiation successful
0806 11:39:16.883873 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":148,"threads_started":1}
I20260806 11:39:16.886401 4044 heartbeater.cc:347] Connected to a master server at 127.21.14.254:37871
I20260806 11:39:16.886674 4044 heartbeater.cc:464] Registering TS with master...
I20260806 11:39:16.887331 4044 heartbeater.cc:511] Master 127.21.14.254:37871 requested a full tablet report, sending...
I20260806 11:39:16.889158 3841 ts_manager.cc:194] Registered new tserver with Master: 6ce2bc1867f74eb6b113467463a9430a (127.21.14.193:43653)
I20260806 11:39:16.890398 3841 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.14.193@KRBTEST.COM'} at 127.21.14.193:38105
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab.
Entry for principal kudu/127.21.14.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab.
I20260806 11:39:16.902665 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.194:0
--local_ip_for_outbound_sockets=127.21.14.194
--webserver_interface=127.21.14.194
--webserver_port=0
--tserver_master_addrs=127.21.14.254:37871
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab
--principal=kudu/127.21.14.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:45535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:39:17.022104 4049 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:39:17.022429 4049 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:39:17.022544 4049 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:39:17.026808 4049 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:39:17.026932 4049 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:39:17.027024 4049 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.194
I20260806 11:39:17.032649 4049 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:45535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab
--principal=kudu/127.21.14.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.14.194
--webserver_port=0
--tserver_master_addrs=127.21.14.254:37871
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4049
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.194
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:39:17.033823 4049 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:39:17.034945 4049 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:39:17.045609 4058 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:39:17.045605 4054 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:39:17.045629 4049 server_base.cc:1034] running on GCE node
W20260806 11:39:17.045601 4055 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:39:17.046271 4049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:39:17.047048 4049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:39:17.048247 4049 hybrid_clock.cc:648] HybridClock initialized: now 1786016357048220 us; error 40 us; skew 500 ppm
Aug 06 11:39:17 dist-test-slave-7c35 krb5kdc[3793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016357, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:39:17.051328 4049 init.cc:377] Logged in from keytab as kudu/127.21.14.194@KRBTEST.COM (short username kudu)
I20260806 11:39:17.052305 4049 webserver.cc:460] Webserver started at http://127.21.14.194:42217/ using document root <none> and password file <none>
I20260806 11:39:17.052872 4049 fs_manager.cc:359] Metadata directory not provided
I20260806 11:39:17.052935 4049 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:39:17.053117 4049 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:39:17.055341 4049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-1/data/instance:
uuid: "32e458de791a4f01ac55419385005fa0"
format_stamp: "Formatted at 2026-08-06 11:39:17 on dist-test-slave-7c35"
server_key: "938466b98f9733c6a445968b4733bdb9"
server_key_iv: "a7e538923c63b93b4c6d1effc5fb4cc6"
server_key_version: "encryptionkey@0"
I20260806 11:39:17.055908 4049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-1/wal/instance:
uuid: "32e458de791a4f01ac55419385005fa0"
format_stamp: "Formatted at 2026-08-06 11:39:17 on dist-test-slave-7c35"
server_key: "938466b98f9733c6a445968b4733bdb9"
server_key_iv: "a7e538923c63b93b4c6d1effc5fb4cc6"
server_key_version: "encryptionkey@0"
I20260806 11:39:17.060169 4049 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 11:39:17.063267 4067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:39:17.064761 4049 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 11:39:17.064945 4049 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-1/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-1/wal
uuid: "32e458de791a4f01ac55419385005fa0"
format_stamp: "Formatted at 2026-08-06 11:39:17 on dist-test-slave-7c35"
server_key: "938466b98f9733c6a445968b4733bdb9"
server_key_iv: "a7e538923c63b93b4c6d1effc5fb4cc6"
server_key_version: "encryptionkey@0"
I20260806 11:39:17.065094 4049 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:39:17.082302 4049 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:39:17.086030 4049 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:39:17.086300 4049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:39:17.087109 4049 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:39:17.088162 4049 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:39:17.088255 4049 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:39:17.088367 4049 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:39:17.088416 4049 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:39:17.099874 4049 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.194:37571
I20260806 11:39:17.100019 4187 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.194:37571 every 8 connection(s)
I20260806 11:39:17.101270 4049 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
I20260806 11:39:17.111479 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 4049
I20260806 11:39:17.111605 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-1/wal/instance
I20260806 11:39:17.111898 21563 external_mini_cluster.cc:1428] Setting key b9ae4c93a5bd19ec8e6fbca16d199793
Aug 06 11:39:17 dist-test-slave-7c35 krb5kdc[3793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016357, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.194@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:39:17.120002 3882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:17.103713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:43903 (local address 127.21.14.254:37871)
0806 11:39:17.103903 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:17.103909 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:17.104926 (+ 1017us) server_negotiation.cc:407] Connection header received
0806 11:39:17.106628 (+ 1702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:17.106632 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:17.106689 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:17.106793 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:39:17.108540 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:17.109111 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:17.110591 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:17.110827 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:17.110999 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:17.114577 (+ 3578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:39:17.114581 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:39:17.114584 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:39:17.114617 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:39:17.116685 (+ 2068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:17.117758 (+ 1073us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:17.117762 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:17.117764 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:17.117855 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:17.118351 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:17.118355 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:17.118356 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:17.118586 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:39:17.118683 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:17.119667 (+ 984us) server_negotiation.cc:299] Negotiation successful
0806 11:39:17.119866 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 11:39:17.121770 4190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:17.104163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:37871 (local address 127.21.14.194:43903)
0806 11:39:17.104732 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:17.104794 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 11:39:17.106321 (+ 1527us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:39:17.107228 (+ 907us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:39:17.107238 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:39:17.107736 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:39:17.108305 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:17.108322 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:17.109281 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:17.109284 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:17.110430 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:17.110443 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:17.111184 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:17.111188 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:17.111381 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:17.112163 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:39:17.112187 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:39:17.114379 (+ 2192us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:39:17.117074 (+ 2695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:39:17.117081 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:39:17.117099 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:39:17.117612 (+ 513us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:39:17.117970 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:39:17.117976 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:39:17.117981 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:39:17.118208 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:39:17.118712 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:39:17.118722 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:39:17.119189 (+ 467us) client_negotiation.cc:769] Sending connection context
0806 11:39:17.119483 (+ 294us) client_negotiation.cc:240] Negotiation successful
0806 11:39:17.119951 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":138,"threads_started":1}
I20260806 11:39:17.123497 4188 heartbeater.cc:347] Connected to a master server at 127.21.14.254:37871
I20260806 11:39:17.123799 4188 heartbeater.cc:464] Registering TS with master...
I20260806 11:39:17.124439 4188 heartbeater.cc:511] Master 127.21.14.254:37871 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.21.14.195@KRBTEST.COM; defaulting to no policy
I20260806 11:39:17.126034 3841 ts_manager.cc:194] Registered new tserver with Master: 32e458de791a4f01ac55419385005fa0 (127.21.14.194:37571)
I20260806 11:39:17.126767 3841 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.14.194@KRBTEST.COM'} at 127.21.14.194:43903
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab.
Entry for principal kudu/127.21.14.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab.
I20260806 11:39:17.151177 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.195:0
--local_ip_for_outbound_sockets=127.21.14.195
--webserver_interface=127.21.14.195
--webserver_port=0
--tserver_master_addrs=127.21.14.254:37871
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab
--principal=kudu/127.21.14.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:45535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:39:17.293205 4193 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:39:17.293640 4193 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:39:17.293735 4193 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:39:17.298030 4193 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:39:17.298136 4193 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:39:17.298234 4193 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.195
I20260806 11:39:17.304057 4193 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:45535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab
--principal=kudu/127.21.14.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.14.195
--webserver_port=0
--tserver_master_addrs=127.21.14.254:37871
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4193
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.195
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:39:17.305290 4193 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:39:17.306406 4193 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:39:17.317188 4199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:39:17.317198 4198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:39:17.317245 4201 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:39:17.317711 4193 server_base.cc:1034] running on GCE node
I20260806 11:39:17.318202 4193 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:39:17.319038 4193 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:39:17.320235 4193 hybrid_clock.cc:648] HybridClock initialized: now 1786016357320195 us; error 52 us; skew 500 ppm
Aug 06 11:39:17 dist-test-slave-7c35 krb5kdc[3793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016357, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:39:17.323661 4193 init.cc:377] Logged in from keytab as kudu/127.21.14.195@KRBTEST.COM (short username kudu)
I20260806 11:39:17.324822 4193 webserver.cc:460] Webserver started at http://127.21.14.195:39387/ using document root <none> and password file <none>
I20260806 11:39:17.325443 4193 fs_manager.cc:359] Metadata directory not provided
I20260806 11:39:17.325515 4193 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:39:17.325775 4193 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:39:17.328250 4193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-2/data/instance:
uuid: "f4fc782bf02c45ea9f7ba7a86862f929"
format_stamp: "Formatted at 2026-08-06 11:39:17 on dist-test-slave-7c35"
server_key: "559653b6b7a910b2ca05056057139afc"
server_key_iv: "0b8e20d7bf224591b44bde6d71b8ab90"
server_key_version: "encryptionkey@0"
I20260806 11:39:17.328851 4193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-2/wal/instance:
uuid: "f4fc782bf02c45ea9f7ba7a86862f929"
format_stamp: "Formatted at 2026-08-06 11:39:17 on dist-test-slave-7c35"
server_key: "559653b6b7a910b2ca05056057139afc"
server_key_iv: "0b8e20d7bf224591b44bde6d71b8ab90"
server_key_version: "encryptionkey@0"
I20260806 11:39:17.333174 4193 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 11:39:17.335994 4208 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:39:17.337416 4193 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 11:39:17.337576 4193 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-2/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-2/wal
uuid: "f4fc782bf02c45ea9f7ba7a86862f929"
format_stamp: "Formatted at 2026-08-06 11:39:17 on dist-test-slave-7c35"
server_key: "559653b6b7a910b2ca05056057139afc"
server_key_iv: "0b8e20d7bf224591b44bde6d71b8ab90"
server_key_version: "encryptionkey@0"
I20260806 11:39:17.337706 4193 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:39:17.357342 4193 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:39:17.360870 4193 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:39:17.361112 4193 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:39:17.361765 4193 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:39:17.362663 4193 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:39:17.362735 4193 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:39:17.362803 4193 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:39:17.362849 4193 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:39:17.372896 4193 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.195:45493
I20260806 11:39:17.372928 4329 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.195:45493 every 8 connection(s)
I20260806 11:39:17.373911 4193 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
I20260806 11:39:17.380229 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 4193
I20260806 11:39:17.380367 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-2/wal/instance
I20260806 11:39:17.380681 21563 external_mini_cluster.cc:1428] Setting key 7fbc799c9d833a98e02f2f4a7d39b0d6
Aug 06 11:39:17 dist-test-slave-7c35 krb5kdc[3793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016357, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.195@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:39:17.391000 3882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:17.376043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:44339 (local address 127.21.14.254:37871)
0806 11:39:17.376206 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:17.376211 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:17.377224 (+ 1013us) server_negotiation.cc:407] Connection header received
0806 11:39:17.378785 (+ 1561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:17.378789 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:17.378843 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:17.378954 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:39:17.380530 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:17.381141 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:17.382693 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:17.383040 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:17.383171 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:17.385969 (+ 2798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:39:17.385972 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:39:17.385975 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:39:17.386004 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:39:17.388096 (+ 2092us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:17.388773 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:17.388793 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:17.388794 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:17.389076 (+ 282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:17.389610 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:17.389615 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:17.389619 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:17.389868 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:39:17.389956 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:17.390646 (+ 690us) server_negotiation.cc:299] Negotiation successful
0806 11:39:17.390784 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 11:39:17.391873 4332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:17.376455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:37871 (local address 127.21.14.195:44339)
0806 11:39:17.377015 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:17.377105 (+ 90us) client_negotiation.cc:174] Beginning negotiation
0806 11:39:17.378548 (+ 1443us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:39:17.379203 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:39:17.379210 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:39:17.379604 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:39:17.380290 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:17.380315 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:17.381300 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:17.381302 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:17.382551 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:17.382563 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:17.383216 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:17.383220 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:17.383398 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:17.384066 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:39:17.384084 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:39:17.385725 (+ 1641us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:39:17.388256 (+ 2531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:39:17.388264 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:39:17.388292 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:39:17.388656 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:39:17.389234 (+ 578us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:39:17.389237 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:39:17.389238 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:39:17.389423 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:39:17.390041 (+ 618us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:39:17.390048 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:39:17.390364 (+ 316us) client_negotiation.cc:769] Sending connection context
0806 11:39:17.390556 (+ 192us) client_negotiation.cc:240] Negotiation successful
0806 11:39:17.390945 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":120,"threads_started":1}
I20260806 11:39:17.393196 4330 heartbeater.cc:347] Connected to a master server at 127.21.14.254:37871
I20260806 11:39:17.393464 4330 heartbeater.cc:464] Registering TS with master...
I20260806 11:39:17.394138 4330 heartbeater.cc:511] Master 127.21.14.254:37871 requested a full tablet report, sending...
I20260806 11:39:17.395378 3841 ts_manager.cc:194] Registered new tserver with Master: f4fc782bf02c45ea9f7ba7a86862f929 (127.21.14.195:45493)
I20260806 11:39:17.395946 3841 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.14.195@KRBTEST.COM'} at 127.21.14.195:44339
I20260806 11:39:17.407546 21563 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 11:39:17.417970 3882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:17.409777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47762 (local address 127.21.14.254:37871)
0806 11:39:17.409928 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:17.409932 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:17.410102 (+ 170us) server_negotiation.cc:407] Connection header received
0806 11:39:17.410201 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:17.410203 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:17.410251 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:17.410327 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:39:17.411316 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:17.411879 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:17.412831 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:17.413070 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:17.413168 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:17.414791 (+ 1623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:39:17.414795 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:39:17.414798 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:39:17.414827 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:39:17.416328 (+ 1501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:17.416765 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:17.416769 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:17.416770 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:17.416855 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:17.417092 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:17.417094 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:17.417095 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:17.417334 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:39:17.417464 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:17.417697 (+ 233us) server_negotiation.cc:299] Negotiation successful
0806 11:39:17.417839 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 11:39:17.422230 3841 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47762:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 11:39:17.425917 3841 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 11:39:17.442951 4347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:17.436969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52226 (local address 127.21.14.195:45493)
0806 11:39:17.437312 (+ 343us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:17.437328 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:17.437351 (+ 23us) server_negotiation.cc:407] Connection header received
0806 11:39:17.437432 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:17.437438 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:17.437648 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:17.438059 (+ 411us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:39:17.438251 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:17.439495 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:17.441181 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:17.442113 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:17.442250 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:17.442511 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:17.442617 (+ 106us) server_negotiation.cc:299] Negotiation successful
0806 11:39:17.442746 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":241,"thread_start_us":145,"threads_started":1}
I20260806 11:39:17.442979 4342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:17.436418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.195:45493 (local address 127.0.0.1:52226)
0806 11:39:17.436878 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:17.436943 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0806 11:39:17.437093 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:39:17.437837 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:39:17.437844 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:39:17.437878 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:39:17.438152 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:17.438161 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:17.439637 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:17.439640 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:17.441049 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:17.441058 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:17.442263 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:17.442266 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:17.442400 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:17.442446 (+ 46us) client_negotiation.cc:769] Sending connection context
0806 11:39:17.442604 (+ 158us) client_negotiation.cc:240] Negotiation successful
0806 11:39:17.442749 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"mutex_wait_us":211,"spinlock_wait_cycles":44800,"thread_start_us":121,"threads_started":1}
I20260806 11:39:17.443614 4343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:17.435848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35580 (local address 127.21.14.194:37571)
0806 11:39:17.436289 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:17.436305 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:17.437250 (+ 945us) server_negotiation.cc:407] Connection header received
0806 11:39:17.437342 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:17.437348 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:17.437601 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:17.437764 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:39:17.438269 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:17.439495 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:17.441422 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:17.442294 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:17.442400 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:17.442551 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:17.443332 (+ 781us) server_negotiation.cc:299] Negotiation successful
0806 11:39:17.443448 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":221,"threads_started":1}
I20260806 11:39:17.444003 4345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:17.435662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.194:37571 (local address 127.0.0.1:35580)
0806 11:39:17.436879 (+ 1217us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:17.436943 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0806 11:39:17.437093 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:39:17.437837 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:39:17.437844 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:39:17.437877 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:39:17.438122 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:17.438132 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:17.439753 (+ 1621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:17.439759 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:17.441246 (+ 1487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:17.441258 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:17.442617 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:17.442620 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:17.442794 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:17.442815 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 11:39:17.442963 (+ 148us) client_negotiation.cc:240] Negotiation successful
0806 11:39:17.443057 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1029,"mutex_wait_us":233,"spinlock_wait_cycles":3072,"thread_start_us":85,"threads_started":1}
I20260806 11:39:17.444183 4346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:17.436674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41410 (local address 127.21.14.193:43653)
0806 11:39:17.437377 (+ 703us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:17.437383 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:17.437618 (+ 235us) server_negotiation.cc:407] Connection header received
0806 11:39:17.437847 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:17.437852 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:17.438052 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:17.438174 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:39:17.438558 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:17.439393 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:17.441384 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:17.442172 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:17.442266 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:17.442421 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:17.442808 (+ 387us) server_negotiation.cc:299] Negotiation successful
0806 11:39:17.442965 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":599,"thread_start_us":136,"threads_started":1}
I20260806 11:39:17.444753 4344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:17.436487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.193:43653 (local address 127.0.0.1:41410)
0806 11:39:17.437524 (+ 1037us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:17.437538 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 11:39:17.437694 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:39:17.438348 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:39:17.438350 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:39:17.438364 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:39:17.438443 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:17.438447 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:17.439507 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:17.439514 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:17.441250 (+ 1736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:17.441258 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:17.442491 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:17.442496 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:17.442607 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:17.442671 (+ 64us) client_negotiation.cc:769] Sending connection context
0806 11:39:17.444541 (+ 1870us) client_negotiation.cc:240] Negotiation successful
0806 11:39:17.444629 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":976,"mutex_wait_us":155,"thread_start_us":114,"threads_started":1}
I20260806 11:39:17.446547 4258 tablet_service.cc:1467] Processing CreateTablet for tablet 314feff7438a46ee927eeade8130d9e0 (DEFAULT_TABLE table=test-table [id=0188b6877aca49169233cd6df4a59e67]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:39:17.447013 4115 tablet_service.cc:1467] Processing CreateTablet for tablet 314feff7438a46ee927eeade8130d9e0 (DEFAULT_TABLE table=test-table [id=0188b6877aca49169233cd6df4a59e67]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:39:17.447715 4258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 314feff7438a46ee927eeade8130d9e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:39:17.447441 3971 tablet_service.cc:1467] Processing CreateTablet for tablet 314feff7438a46ee927eeade8130d9e0 (DEFAULT_TABLE table=test-table [id=0188b6877aca49169233cd6df4a59e67]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:39:17.448094 4115 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 314feff7438a46ee927eeade8130d9e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:39:17.448426 3971 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 314feff7438a46ee927eeade8130d9e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:39:17.454321 4348 tablet_bootstrap.cc:492] T 314feff7438a46ee927eeade8130d9e0 P f4fc782bf02c45ea9f7ba7a86862f929: Bootstrap starting.
I20260806 11:39:17.456444 4349 tablet_bootstrap.cc:492] T 314feff7438a46ee927eeade8130d9e0 P 6ce2bc1867f74eb6b113467463a9430a: Bootstrap starting.
I20260806 11:39:17.456945 4348 tablet_bootstrap.cc:654] T 314feff7438a46ee927eeade8130d9e0 P f4fc782bf02c45ea9f7ba7a86862f929: Neither blocks nor log segments found. Creating new log.
I20260806 11:39:17.458109 4348 log.cc:824] T 314feff7438a46ee927eeade8130d9e0 P f4fc782bf02c45ea9f7ba7a86862f929: Log is configured to *not* fsync() on all Append() calls
I20260806 11:39:17.458547 4349 tablet_bootstrap.cc:654] T 314feff7438a46ee927eeade8130d9e0 P 6ce2bc1867f74eb6b113467463a9430a: Neither blocks nor log segments found. Creating new log.
I20260806 11:39:17.459275 4351 tablet_bootstrap.cc:492] T 314feff7438a46ee927eeade8130d9e0 P 32e458de791a4f01ac55419385005fa0: Bootstrap starting.
I20260806 11:39:17.459549 4349 log.cc:824] T 314feff7438a46ee927eeade8130d9e0 P 6ce2bc1867f74eb6b113467463a9430a: Log is configured to *not* fsync() on all Append() calls
I20260806 11:39:17.460618 4348 tablet_bootstrap.cc:492] T 314feff7438a46ee927eeade8130d9e0 P f4fc782bf02c45ea9f7ba7a86862f929: No bootstrap required, opened a new log
I20260806 11:39:17.460851 4348 ts_tablet_manager.cc:1397] T 314feff7438a46ee927eeade8130d9e0 P f4fc782bf02c45ea9f7ba7a86862f929: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260806 11:39:17.461818 4349 tablet_bootstrap.cc:492] T 314feff7438a46ee927eeade8130d9e0 P 6ce2bc1867f74eb6b113467463a9430a: No bootstrap required, opened a new log
I20260806 11:39:17.462121 4351 tablet_bootstrap.cc:654] T 314feff7438a46ee927eeade8130d9e0 P 32e458de791a4f01ac55419385005fa0: Neither blocks nor log segments found. Creating new log.
I20260806 11:39:17.462143 4349 ts_tablet_manager.cc:1397] T 314feff7438a46ee927eeade8130d9e0 P 6ce2bc1867f74eb6b113467463a9430a: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260806 11:39:17.463136 4351 log.cc:824] T 314feff7438a46ee927eeade8130d9e0 P 32e458de791a4f01ac55419385005fa0: Log is configured to *not* fsync() on all Append() calls
I20260806 11:39:17.464553 4348 raft_consensus.cc:348] T 314feff7438a46ee927eeade8130d9e0 P f4fc782bf02c45ea9f7ba7a86862f929 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32e458de791a4f01ac55419385005fa0" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37571 } } peers { permanent_uuid: "f4fc782bf02c45ea9f7ba7a86862f929" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 45493 } } peers { permanent_uuid: "6ce2bc1867f74eb6b113467463a9430a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 43653 } }
I20260806 11:39:17.464807 4348 raft_consensus.cc:374] T 314feff7438a46ee927eeade8130d9e0 P f4fc782bf02c45ea9f7ba7a86862f929 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:39:17.464893 4348 raft_consensus.cc:729] T 314feff7438a46ee927eeade8130d9e0 P f4fc782bf02c45ea9f7ba7a86862f929 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4fc782bf02c45ea9f7ba7a86862f929, State: Initialized, Role: FOLLOWER
I20260806 11:39:17.465387 4348 consensus_queue.cc:260] T 314feff7438a46ee927eeade8130d9e0 P f4fc782bf02c45ea9f7ba7a86862f929 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32e458de791a4f01ac55419385005fa0" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37571 } } peers { permanent_uuid: "f4fc782bf02c45ea9f7ba7a86862f929" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 45493 } } peers { permanent_uuid: "6ce2bc1867f74eb6b113467463a9430a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 43653 } }
I20260806 11:39:17.465478 4351 tablet_bootstrap.cc:492] T 314feff7438a46ee927eeade8130d9e0 P 32e458de791a4f01ac55419385005fa0: No bootstrap required, opened a new log
I20260806 11:39:17.465677 4351 ts_tablet_manager.cc:1397] T 314feff7438a46ee927eeade8130d9e0 P 32e458de791a4f01ac55419385005fa0: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 11:39:17.466327 4348 ts_tablet_manager.cc:1428] T 314feff7438a46ee927eeade8130d9e0 P f4fc782bf02c45ea9f7ba7a86862f929: Time spent starting tablet: real 0.005s user 0.007s sys 0.000s
I20260806 11:39:17.466727 4330 heartbeater.cc:503] Master 127.21.14.254:37871 was elected leader, sending a full tablet report...
I20260806 11:39:17.466779 4349 raft_consensus.cc:348] T 314feff7438a46ee927eeade8130d9e0 P 6ce2bc1867f74eb6b113467463a9430a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32e458de791a4f01ac55419385005fa0" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37571 } } peers { permanent_uuid: "f4fc782bf02c45ea9f7ba7a86862f929" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 45493 } } peers { permanent_uuid: "6ce2bc1867f74eb6b113467463a9430a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 43653 } }
I20260806 11:39:17.467093 4349 raft_consensus.cc:374] T 314feff7438a46ee927eeade8130d9e0 P 6ce2bc1867f74eb6b113467463a9430a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:39:17.467151 4349 raft_consensus.cc:729] T 314feff7438a46ee927eeade8130d9e0 P 6ce2bc1867f74eb6b113467463a9430a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ce2bc1867f74eb6b113467463a9430a, State: Initialized, Role: FOLLOWER
I20260806 11:39:17.467612 4349 consensus_queue.cc:260] T 314feff7438a46ee927eeade8130d9e0 P 6ce2bc1867f74eb6b113467463a9430a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32e458de791a4f01ac55419385005fa0" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37571 } } peers { permanent_uuid: "f4fc782bf02c45ea9f7ba7a86862f929" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 45493 } } peers { permanent_uuid: "6ce2bc1867f74eb6b113467463a9430a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 43653 } }
I20260806 11:39:17.468986 4349 ts_tablet_manager.cc:1428] T 314feff7438a46ee927eeade8130d9e0 P 6ce2bc1867f74eb6b113467463a9430a: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260806 11:39:17.469436 4044 heartbeater.cc:503] Master 127.21.14.254:37871 was elected leader, sending a full tablet report...
I20260806 11:39:17.469756 4351 raft_consensus.cc:348] T 314feff7438a46ee927eeade8130d9e0 P 32e458de791a4f01ac55419385005fa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32e458de791a4f01ac55419385005fa0" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37571 } } peers { permanent_uuid: "f4fc782bf02c45ea9f7ba7a86862f929" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 45493 } } peers { permanent_uuid: "6ce2bc1867f74eb6b113467463a9430a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 43653 } }
I20260806 11:39:17.470026 4351 raft_consensus.cc:374] T 314feff7438a46ee927eeade8130d9e0 P 32e458de791a4f01ac55419385005fa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:39:17.470100 4351 raft_consensus.cc:729] T 314feff7438a46ee927eeade8130d9e0 P 32e458de791a4f01ac55419385005fa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32e458de791a4f01ac55419385005fa0, State: Initialized, Role: FOLLOWER
I20260806 11:39:17.470568 4351 consensus_queue.cc:260] T 314feff7438a46ee927eeade8130d9e0 P 32e458de791a4f01ac55419385005fa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32e458de791a4f01ac55419385005fa0" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37571 } } peers { permanent_uuid: "f4fc782bf02c45ea9f7ba7a86862f929" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 45493 } } peers { permanent_uuid: "6ce2bc1867f74eb6b113467463a9430a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 43653 } }
I20260806 11:39:17.471671 4351 ts_tablet_manager.cc:1428] T 314feff7438a46ee927eeade8130d9e0 P 32e458de791a4f01ac55419385005fa0: Time spent starting tablet: real 0.006s user 0.000s sys 0.005s
I20260806 11:39:17.472059 4188 heartbeater.cc:503] Master 127.21.14.254:37871 was elected leader, sending a full tablet report...
W20260806 11:39:17.602818 4189 tablet.cc:2369] T 314feff7438a46ee927eeade8130d9e0 P 32e458de791a4f01ac55419385005fa0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 11:39:17.618206 4045 tablet.cc:2369] T 314feff7438a46ee927eeade8130d9e0 P 6ce2bc1867f74eb6b113467463a9430a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 11:39:17.624991 4331 tablet.cc:2369] T 314feff7438a46ee927eeade8130d9e0 P f4fc782bf02c45ea9f7ba7a86862f929: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 11:39:17.738426 4356 raft_consensus.cc:482] T 314feff7438a46ee927eeade8130d9e0 P f4fc782bf02c45ea9f7ba7a86862f929 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 11:39:17.738662 4356 raft_consensus.cc:504] T 314feff7438a46ee927eeade8130d9e0 P f4fc782bf02c45ea9f7ba7a86862f929 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32e458de791a4f01ac55419385005fa0" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37571 } } peers { permanent_uuid: "f4fc782bf02c45ea9f7ba7a86862f929" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 45493 } } peers { permanent_uuid: "6ce2bc1867f74eb6b113467463a9430a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 43653 } }
I20260806 11:39:17.740020 4356 leader_election.cc:288] T 314feff7438a46ee927eeade8130d9e0 P f4fc782bf02c45ea9f7ba7a86862f929 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32e458de791a4f01ac55419385005fa0 (127.21.14.194:37571), 6ce2bc1867f74eb6b113467463a9430a (127.21.14.193:43653)
I20260806 11:39:17.744457 4343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:17.740514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:34811 (local address 127.21.14.194:37571)
0806 11:39:17.740701 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:17.740706 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:17.740724 (+ 18us) server_negotiation.cc:407] Connection header received
0806 11:39:17.740793 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:17.740796 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:17.740842 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:17.740951 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:39:17.741252 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:17.741917 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:17.743238 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:17.743774 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:17.743879 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:17.743933 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:17.744228 (+ 295us) server_negotiation.cc:299] Negotiation successful
0806 11:39:17.744296 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260806 11:39:17.744475 4332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:17.740250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.194:37571 (local address 127.21.14.195:34811)
0806 11:39:17.740433 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:17.740450 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 11:39:17.740558 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:39:17.740965 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:39:17.740969 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:39:17.740987 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:39:17.741121 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:17.741127 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:17.742025 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:17.742030 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:17.743106 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:17.743117 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:17.743930 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:17.743935 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:17.744062 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:17.744094 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 11:39:17.744212 (+ 118us) client_negotiation.cc:240] Negotiation successful
0806 11:39:17.744304 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86,"mutex_wait_us":86}
I20260806 11:39:17.744951 4346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:17.740530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:38137 (local address 127.21.14.193:43653)
0806 11:39:17.740709 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:17.740714 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:17.740924 (+ 210us) server_negotiation.cc:407] Connection header received
0806 11:39:17.741079 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:17.741081 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:17.741125 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:17.741204 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:39:17.741519 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:17.742014 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:17.743621 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:17.744271 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:17.744410 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:17.744464 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:17.744726 (+ 262us) server_negotiation.cc:299] Negotiation successful
0806 11:39:17.744831 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260806 11:39:17.745298 4359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:17.740445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.193:43653 (local address 127.21.14.195:38137)
0806 11:39:17.740835 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:17.740845 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 11:39:17.740961 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:39:17.741227 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:39:17.741230 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:39:17.741245 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:39:17.741370 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:17.741375 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:17.742139 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:17.742145 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:17.743462 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:17.743474 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:17.744429 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:17.744434 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:17.744546 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:17.744580 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 11:39:17.745108 (+ 528us) client_negotiation.cc:240] Negotiation successful
0806 11:39:17.745194 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":159,"threads_started":1}
I20260806 11:39:17.745390 4139 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "314feff7438a46ee927eeade8130d9e0" candidate_uuid: "f4fc782bf02c45ea9f7ba7a86862f929" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32e458de791a4f01ac55419385005fa0" is_pre_election: true
I20260806 11:39:17.745806 4139 raft_consensus.cc:2454] T 314feff7438a46ee927eeade8130d9e0 P 32e458de791a4f01ac55419385005fa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4fc782bf02c45ea9f7ba7a86862f929 in term 0.
I20260806 11:39:17.745972 3994 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "314feff7438a46ee927eeade8130d9e0" candidate_uuid: "f4fc782bf02c45ea9f7ba7a86862f929" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ce2bc1867f74eb6b113467463a9430a" is_pre_election: true
I20260806 11:39:17.746289 3994 raft_consensus.cc:2454] T 314feff7438a46ee927eeade8130d9e0 P 6ce2bc1867f74eb6b113467463a9430a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4fc782bf02c45ea9f7ba7a86862f929 in term 0.
I20260806 11:39:17.746464 4212 leader_election.cc:302] T 314feff7438a46ee927eeade8130d9e0 P f4fc782bf02c45ea9f7ba7a86862f929 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32e458de791a4f01ac55419385005fa0, f4fc782bf02c45ea9f7ba7a86862f929; no voters:
I20260806 11:39:17.746793 4356 raft_consensus.cc:2781] T 314feff7438a46ee927eeade8130d9e0 P f4fc782bf02c45ea9f7ba7a86862f929 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 11:39:17.746907 4356 raft_consensus.cc:482] T 314feff7438a46ee927eeade8130d9e0 P f4fc782bf02c45ea9f7ba7a86862f929 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 11:39:17.746963 4356 raft_consensus.cc:3037] T 314feff7438a46ee927eeade8130d9e0 P f4fc782bf02c45ea9f7ba7a86862f929 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:39:17.748093 4356 raft_consensus.cc:504] T 314feff7438a46ee927eeade8130d9e0 P f4fc782bf02c45ea9f7ba7a86862f929 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32e458de791a4f01ac55419385005fa0" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37571 } } peers { permanent_uuid: "f4fc782bf02c45ea9f7ba7a86862f929" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 45493 } } peers { permanent_uuid: "6ce2bc1867f74eb6b113467463a9430a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 43653 } }
I20260806 11:39:17.748466 4356 leader_election.cc:288] T 314feff7438a46ee927eeade8130d9e0 P f4fc782bf02c45ea9f7ba7a86862f929 [CANDIDATE]: Term 1 election: Requested vote from peers 32e458de791a4f01ac55419385005fa0 (127.21.14.194:37571), 6ce2bc1867f74eb6b113467463a9430a (127.21.14.193:43653)
I20260806 11:39:17.748850 4139 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "314feff7438a46ee927eeade8130d9e0" candidate_uuid: "f4fc782bf02c45ea9f7ba7a86862f929" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32e458de791a4f01ac55419385005fa0"
I20260806 11:39:17.748924 3994 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "314feff7438a46ee927eeade8130d9e0" candidate_uuid: "f4fc782bf02c45ea9f7ba7a86862f929" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ce2bc1867f74eb6b113467463a9430a"
I20260806 11:39:17.749010 4139 raft_consensus.cc:3037] T 314feff7438a46ee927eeade8130d9e0 P 32e458de791a4f01ac55419385005fa0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:39:17.749078 3994 raft_consensus.cc:3037] T 314feff7438a46ee927eeade8130d9e0 P 6ce2bc1867f74eb6b113467463a9430a [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:39:17.750147 4139 raft_consensus.cc:2454] T 314feff7438a46ee927eeade8130d9e0 P 32e458de791a4f01ac55419385005fa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4fc782bf02c45ea9f7ba7a86862f929 in term 1.
I20260806 11:39:17.750238 3994 raft_consensus.cc:2454] T 314feff7438a46ee927eeade8130d9e0 P 6ce2bc1867f74eb6b113467463a9430a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4fc782bf02c45ea9f7ba7a86862f929 in term 1.
I20260806 11:39:17.750500 4212 leader_election.cc:302] T 314feff7438a46ee927eeade8130d9e0 P f4fc782bf02c45ea9f7ba7a86862f929 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32e458de791a4f01ac55419385005fa0, f4fc782bf02c45ea9f7ba7a86862f929; no voters:
I20260806 11:39:17.750680 4356 raft_consensus.cc:2781] T 314feff7438a46ee927eeade8130d9e0 P f4fc782bf02c45ea9f7ba7a86862f929 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:39:17.750955 4356 raft_consensus.cc:686] T 314feff7438a46ee927eeade8130d9e0 P f4fc782bf02c45ea9f7ba7a86862f929 [term 1 LEADER]: Becoming Leader. State: Replica: f4fc782bf02c45ea9f7ba7a86862f929, State: Running, Role: LEADER
I20260806 11:39:17.751325 4356 consensus_queue.cc:237] T 314feff7438a46ee927eeade8130d9e0 P f4fc782bf02c45ea9f7ba7a86862f929 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32e458de791a4f01ac55419385005fa0" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37571 } } peers { permanent_uuid: "f4fc782bf02c45ea9f7ba7a86862f929" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 45493 } } peers { permanent_uuid: "6ce2bc1867f74eb6b113467463a9430a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 43653 } }
I20260806 11:39:17.755004 3841 catalog_manager.cc:5495] T 314feff7438a46ee927eeade8130d9e0 P f4fc782bf02c45ea9f7ba7a86862f929 reported cstate change: term changed from 0 to 1, leader changed from <none> to f4fc782bf02c45ea9f7ba7a86862f929 (127.21.14.195). New cstate: current_term: 1 leader_uuid: "f4fc782bf02c45ea9f7ba7a86862f929" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32e458de791a4f01ac55419385005fa0" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f4fc782bf02c45ea9f7ba7a86862f929" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 45493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ce2bc1867f74eb6b113467463a9430a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 43653 } health_report { overall_health: UNKNOWN } } }
I20260806 11:39:17.812332 4347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:17.808478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52238 (local address 127.21.14.195:45493)
0806 11:39:17.808617 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:17.808622 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:17.808638 (+ 16us) server_negotiation.cc:407] Connection header received
0806 11:39:17.808748 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:17.808750 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:17.808808 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:17.808928 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 11:39:17.809218 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:17.809767 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:17.810688 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:17.810940 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:17.811057 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:17.811404 (+ 347us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 11:39:17.811877 (+ 473us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 11:39:17.812016 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:17.812087 (+ 71us) server_negotiation.cc:299] Negotiation successful
0806 11:39:17.812155 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 11:39:17.819182 4139 raft_consensus.cc:1261] T 314feff7438a46ee927eeade8130d9e0 P 32e458de791a4f01ac55419385005fa0 [term 1 FOLLOWER]: Refusing update from remote peer f4fc782bf02c45ea9f7ba7a86862f929: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 11:39:17.819229 3994 raft_consensus.cc:1261] T 314feff7438a46ee927eeade8130d9e0 P 6ce2bc1867f74eb6b113467463a9430a [term 1 FOLLOWER]: Refusing update from remote peer f4fc782bf02c45ea9f7ba7a86862f929: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 11:39:17.820056 4360 consensus_queue.cc:1035] T 314feff7438a46ee927eeade8130d9e0 P f4fc782bf02c45ea9f7ba7a86862f929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32e458de791a4f01ac55419385005fa0" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 11:39:17.820302 4356 consensus_queue.cc:1035] T 314feff7438a46ee927eeade8130d9e0 P f4fc782bf02c45ea9f7ba7a86862f929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ce2bc1867f74eb6b113467463a9430a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 43653 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 11:39:17.829100 4364 mvcc.cc:205] Tried to move back new op lower bound from 7315523001618735104 to 7315523001352085504. Current Snapshot: MvccSnapshot[applied={T|T < 7315523001618735104}]
I20260806 11:39:17.829766 4365 mvcc.cc:205] Tried to move back new op lower bound from 7315523001618735104 to 7315523001352085504. Current Snapshot: MvccSnapshot[applied={T|T < 7315523001618735104}]
I20260806 11:39:17.839697 3839 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47762:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 11:39:17.839903 3839 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47762:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 11:39:17.842985 3839 catalog_manager.cc:5782] T 00000000000000000000000000000000 P beea7743cac049cfb113cc825011d29e: Sending DeleteTablet for 3 replicas of tablet 314feff7438a46ee927eeade8130d9e0
I20260806 11:39:17.844043 3971 tablet_service.cc:1514] Processing DeleteTablet for tablet 314feff7438a46ee927eeade8130d9e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:39:17 UTC) from {username='kudu', principal='kudu/127.21.14.254@KRBTEST.COM'} at 127.0.0.1:41410
I20260806 11:39:17.844043 4258 tablet_service.cc:1514] Processing DeleteTablet for tablet 314feff7438a46ee927eeade8130d9e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:39:17 UTC) from {username='kudu', principal='kudu/127.21.14.254@KRBTEST.COM'} at 127.0.0.1:52226
I20260806 11:39:17.844707 4115 tablet_service.cc:1514] Processing DeleteTablet for tablet 314feff7438a46ee927eeade8130d9e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:39:17 UTC) from {username='kudu', principal='kudu/127.21.14.254@KRBTEST.COM'} at 127.0.0.1:35580
I20260806 11:39:17.844908 4375 tablet_replica.cc:330] T 314feff7438a46ee927eeade8130d9e0 P 6ce2bc1867f74eb6b113467463a9430a: stopping tablet replica
I20260806 11:39:17.845228 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 3906
I20260806 11:39:17.846698 4376 tablet_replica.cc:330] T 314feff7438a46ee927eeade8130d9e0 P f4fc782bf02c45ea9f7ba7a86862f929: stopping tablet replica
I20260806 11:39:17.845312 4377 tablet_replica.cc:330] T 314feff7438a46ee927eeade8130d9e0 P 32e458de791a4f01ac55419385005fa0: stopping tablet replica
I20260806 11:39:17.847142 4376 raft_consensus.cc:2229] T 314feff7438a46ee927eeade8130d9e0 P f4fc782bf02c45ea9f7ba7a86862f929 [term 1 LEADER]: Raft consensus shutting down.
I20260806 11:39:17.847357 4377 raft_consensus.cc:2229] T 314feff7438a46ee927eeade8130d9e0 P 32e458de791a4f01ac55419385005fa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 11:39:17.847667 4376 raft_consensus.cc:2258] T 314feff7438a46ee927eeade8130d9e0 P f4fc782bf02c45ea9f7ba7a86862f929 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 11:39:17.847820 4377 raft_consensus.cc:2258] T 314feff7438a46ee927eeade8130d9e0 P 32e458de791a4f01ac55419385005fa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 11:39:17.849725 4376 ts_tablet_manager.cc:1905] T 314feff7438a46ee927eeade8130d9e0 P f4fc782bf02c45ea9f7ba7a86862f929: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 11:39:17.849725 4377 ts_tablet_manager.cc:1905] T 314feff7438a46ee927eeade8130d9e0 P 32e458de791a4f01ac55419385005fa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 11:39:17.852732 4376 ts_tablet_manager.cc:1918] T 314feff7438a46ee927eeade8130d9e0 P f4fc782bf02c45ea9f7ba7a86862f929: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 11:39:17.852732 4377 ts_tablet_manager.cc:1918] T 314feff7438a46ee927eeade8130d9e0 P 32e458de791a4f01ac55419385005fa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 11:39:17.852857 4377 log.cc:1196] T 314feff7438a46ee927eeade8130d9e0 P 32e458de791a4f01ac55419385005fa0: Deleting WAL directory at /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-1/wal/wals/314feff7438a46ee927eeade8130d9e0
I20260806 11:39:17.853004 4376 log.cc:1196] T 314feff7438a46ee927eeade8130d9e0 P f4fc782bf02c45ea9f7ba7a86862f929: Deleting WAL directory at /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-2/wal/wals/314feff7438a46ee927eeade8130d9e0
I20260806 11:39:17.853288 4377 ts_tablet_manager.cc:1939] T 314feff7438a46ee927eeade8130d9e0 P 32e458de791a4f01ac55419385005fa0: Deleting consensus metadata
I20260806 11:39:17.853426 4376 ts_tablet_manager.cc:1939] T 314feff7438a46ee927eeade8130d9e0 P f4fc782bf02c45ea9f7ba7a86862f929: Deleting consensus metadata
I20260806 11:39:17.854874 3825 catalog_manager.cc:4841] TS 32e458de791a4f01ac55419385005fa0 (127.21.14.194:37571): tablet 314feff7438a46ee927eeade8130d9e0 (table test-table [id=0188b6877aca49169233cd6df4a59e67]) successfully deleted
I20260806 11:39:17.854835 3823 catalog_manager.cc:4841] TS f4fc782bf02c45ea9f7ba7a86862f929 (127.21.14.195:45493): tablet 314feff7438a46ee927eeade8130d9e0 (table test-table [id=0188b6877aca49169233cd6df4a59e67]) successfully deleted
W20260806 11:39:17.857167 3822 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.14.193:43653 (error 108)
I20260806 11:39:17.858152 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 4049
I20260806 11:39:17.858392 4344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:17.858055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.193:43653 (local address 127.0.0.1:41422)
0806 11:39:17.858220 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:17.858297 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.193:43653: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260806 11:39:17.864288 3822 catalog_manager.cc:4584] TS 6ce2bc1867f74eb6b113467463a9430a (127.21.14.193:43653): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 314feff7438a46ee927eeade8130d9e0: Network error: Client connection negotiation failed: client connection to 127.21.14.193:43653: connect: Connection refused (error 111)
I20260806 11:39:17.869882 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 4193
I20260806 11:39:17.878404 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 3806
I20260806 11:39:17.887817 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:37871
--webserver_interface=127.21.14.254
--webserver_port=38567
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab
--principal=kudu/127.21.14.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:45535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.14.254:37871
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:39:18.007465 4379 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:39:18.007937 4379 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:39:18.008044 4379 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:39:18.012606 4379 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 11:39:18.012697 4379 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:39:18.012718 4379 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:39:18.012735 4379 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 11:39:18.012751 4379 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 11:39:18.018663 4379 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:45535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.21.14.254:37871
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab
--principal=kudu/127.21.14.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:37871
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.14.254
--webserver_port=38567
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4379
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:39:18.019980 4379 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:39:18.021095 4379 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:39:18.036301 4385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:39:18.036288 4386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:39:18.036587 4379 server_base.cc:1034] running on GCE node
W20260806 11:39:18.036288 4388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:39:18.037433 4379 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:39:18.038813 4379 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:39:18.040055 4379 hybrid_clock.cc:648] HybridClock initialized: now 1786016358040037 us; error 34 us; skew 500 ppm
Aug 06 11:39:18 dist-test-slave-7c35 krb5kdc[3793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016358, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:39:18.043206 4379 init.cc:377] Logged in from keytab as kudu/127.21.14.254@KRBTEST.COM (short username kudu)
I20260806 11:39:18.044266 4379 webserver.cc:460] Webserver started at http://127.21.14.254:38567/ using document root <none> and password file <none>
I20260806 11:39:18.044912 4379 fs_manager.cc:359] Metadata directory not provided
I20260806 11:39:18.045025 4379 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:39:18.049412 4379 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260806 11:39:18.051839 4396 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:39:18.053117 4379 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 11:39:18.053277 4379 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/master-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/master-0/wal
uuid: "beea7743cac049cfb113cc825011d29e"
format_stamp: "Formatted at 2026-08-06 11:39:16 on dist-test-slave-7c35"
server_key: "c8ab8bf50e62fe9626dfaa8e07ca149d"
server_key_iv: "05fa07ec7bb16e4ef5d888862e737b13"
server_key_version: "encryptionkey@0"
I20260806 11:39:18.053933 4379 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:39:18.077355 4379 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:39:18.081059 4379 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:39:18.081295 4379 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:39:18.089138 4379 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.254:37871
I20260806 11:39:18.089468 4454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.254:37871 every 8 connection(s)
I20260806 11:39:18.090422 4379 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
I20260806 11:39:18.095734 4455 sys_catalog.cc:263] Verifying existing consensus state
I20260806 11:39:18.096499 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 4379
I20260806 11:39:18.097007 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.193:43653
--local_ip_for_outbound_sockets=127.21.14.193
--tserver_master_addrs=127.21.14.254:37871
--webserver_port=35725
--webserver_interface=127.21.14.193
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab
--principal=kudu/127.21.14.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:45535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 11:39:18.097862 4455 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P beea7743cac049cfb113cc825011d29e: Bootstrap starting.
I20260806 11:39:18.117082 4455 log.cc:824] T 00000000000000000000000000000000 P beea7743cac049cfb113cc825011d29e: Log is configured to *not* fsync() on all Append() calls
I20260806 11:39:18.126500 4455 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P beea7743cac049cfb113cc825011d29e: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 11:39:18.127020 4455 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P beea7743cac049cfb113cc825011d29e: Bootstrap complete.
I20260806 11:39:18.131667 4455 raft_consensus.cc:348] T 00000000000000000000000000000000 P beea7743cac049cfb113cc825011d29e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beea7743cac049cfb113cc825011d29e" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 37871 } }
I20260806 11:39:18.132228 4455 raft_consensus.cc:729] T 00000000000000000000000000000000 P beea7743cac049cfb113cc825011d29e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: beea7743cac049cfb113cc825011d29e, State: Initialized, Role: FOLLOWER
I20260806 11:39:18.132781 4455 consensus_queue.cc:260] T 00000000000000000000000000000000 P beea7743cac049cfb113cc825011d29e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beea7743cac049cfb113cc825011d29e" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 37871 } }
I20260806 11:39:18.132936 4455 raft_consensus.cc:388] T 00000000000000000000000000000000 P beea7743cac049cfb113cc825011d29e [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 11:39:18.133033 4455 raft_consensus.cc:482] T 00000000000000000000000000000000 P beea7743cac049cfb113cc825011d29e [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 11:39:18.133153 4455 raft_consensus.cc:3037] T 00000000000000000000000000000000 P beea7743cac049cfb113cc825011d29e [term 1 FOLLOWER]: Advancing to term 2
I20260806 11:39:18.135435 4455 raft_consensus.cc:504] T 00000000000000000000000000000000 P beea7743cac049cfb113cc825011d29e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beea7743cac049cfb113cc825011d29e" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 37871 } }
I20260806 11:39:18.135821 4455 leader_election.cc:302] T 00000000000000000000000000000000 P beea7743cac049cfb113cc825011d29e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: beea7743cac049cfb113cc825011d29e; no voters:
I20260806 11:39:18.136188 4455 leader_election.cc:288] T 00000000000000000000000000000000 P beea7743cac049cfb113cc825011d29e [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 11:39:18.136302 4459 raft_consensus.cc:2781] T 00000000000000000000000000000000 P beea7743cac049cfb113cc825011d29e [term 2 FOLLOWER]: Leader election won for term 2
I20260806 11:39:18.136492 4459 raft_consensus.cc:686] T 00000000000000000000000000000000 P beea7743cac049cfb113cc825011d29e [term 2 LEADER]: Becoming Leader. State: Replica: beea7743cac049cfb113cc825011d29e, State: Running, Role: LEADER
I20260806 11:39:18.136817 4459 consensus_queue.cc:237] T 00000000000000000000000000000000 P beea7743cac049cfb113cc825011d29e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beea7743cac049cfb113cc825011d29e" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 37871 } }
I20260806 11:39:18.137744 4455 sys_catalog.cc:564] T 00000000000000000000000000000000 P beea7743cac049cfb113cc825011d29e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 11:39:18.138029 4459 sys_catalog.cc:455] T 00000000000000000000000000000000 P beea7743cac049cfb113cc825011d29e [sys.catalog]: SysCatalogTable state changed. Reason: New leader beea7743cac049cfb113cc825011d29e. Latest consensus state: current_term: 2 leader_uuid: "beea7743cac049cfb113cc825011d29e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beea7743cac049cfb113cc825011d29e" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 37871 } } }
I20260806 11:39:18.138157 4459 sys_catalog.cc:458] T 00000000000000000000000000000000 P beea7743cac049cfb113cc825011d29e [sys.catalog]: This master's current role is: LEADER
I20260806 11:39:18.138581 4463 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 11:39:18.138607 4460 sys_catalog.cc:455] T 00000000000000000000000000000000 P beea7743cac049cfb113cc825011d29e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "beea7743cac049cfb113cc825011d29e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beea7743cac049cfb113cc825011d29e" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 37871 } } }
I20260806 11:39:18.138885 4460 sys_catalog.cc:458] T 00000000000000000000000000000000 P beea7743cac049cfb113cc825011d29e [sys.catalog]: This master's current role is: LEADER
I20260806 11:39:18.144152 4463 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 11:39:18.144929 4463 catalog_manager.cc:1259] Loaded cluster ID: 29f8b5ac6b7845f9a0cc9327663aa7ac
I20260806 11:39:18.145007 4463 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 11:39:18.146544 4463 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260806 11:39:18.151732 4463 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260806 11:39:18.158111 4463 catalog_manager.cc:1504] Loading token signing keys...
I20260806 11:39:18.159122 4463 catalog_manager.cc:5879] T 00000000000000000000000000000000 P beea7743cac049cfb113cc825011d29e: Loaded TSK: 0
I20260806 11:39:18.160415 4463 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 11:39:18.227631 4457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:39:18.227984 4457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:39:18.228070 4457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:39:18.232613 4457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:39:18.232714 4457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:39:18.232813 4457 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.193
I20260806 11:39:18.238669 4457 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:45535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab
--principal=kudu/127.21.14.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.193:43653
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.14.193
--webserver_port=35725
--tserver_master_addrs=127.21.14.254:37871
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4457
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.193
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:39:18.239940 4457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:39:18.241086 4457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:39:18.252054 4485 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:39:18.252079 4484 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:39:18.252079 4487 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:39:18.253284 4457 server_base.cc:1034] running on GCE node
I20260806 11:39:18.253784 4457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:39:18.254614 4457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:39:18.255815 4457 hybrid_clock.cc:648] HybridClock initialized: now 1786016358255789 us; error 77 us; skew 500 ppm
Aug 06 11:39:18 dist-test-slave-7c35 krb5kdc[3793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016358, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:39:18.259061 4457 init.cc:377] Logged in from keytab as kudu/127.21.14.193@KRBTEST.COM (short username kudu)
I20260806 11:39:18.260291 4457 webserver.cc:460] Webserver started at http://127.21.14.193:35725/ using document root <none> and password file <none>
I20260806 11:39:18.260926 4457 fs_manager.cc:359] Metadata directory not provided
I20260806 11:39:18.261040 4457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:39:18.265722 4457 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 11:39:18.268510 4494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:39:18.270043 4457 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 11:39:18.270208 4457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-0/wal
uuid: "6ce2bc1867f74eb6b113467463a9430a"
format_stamp: "Formatted at 2026-08-06 11:39:16 on dist-test-slave-7c35"
server_key: "26832165400930665ce02c708c257e99"
server_key_iv: "8493c9f433424ba15d78243c5433e296"
server_key_version: "encryptionkey@0"
I20260806 11:39:18.270895 4457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:39:18.288391 4457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:39:18.293074 4457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:39:18.293327 4457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:39:18.294081 4457 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:39:18.295646 4501 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 11:39:18.300380 4457 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 11:39:18.300480 4457 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.001s
I20260806 11:39:18.300570 4457 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 11:39:18.302464 4457 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 11:39:18.302523 4457 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260806 11:39:18.302984 4501 tablet_bootstrap.cc:492] T 314feff7438a46ee927eeade8130d9e0 P 6ce2bc1867f74eb6b113467463a9430a: Bootstrap starting.
I20260806 11:39:18.313618 4457 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.193:43653
I20260806 11:39:18.313711 4612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.193:43653 every 8 connection(s)
I20260806 11:39:18.314779 4457 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
I20260806 11:39:18.315550 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 4457
I20260806 11:39:18.316097 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.194:37571
--local_ip_for_outbound_sockets=127.21.14.194
--tserver_master_addrs=127.21.14.254:37871
--webserver_port=42217
--webserver_interface=127.21.14.194
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab
--principal=kudu/127.21.14.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:45535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 11:39:18 dist-test-slave-7c35 krb5kdc[3793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016358, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.193@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:39:18.327745 4501 log.cc:824] T 314feff7438a46ee927eeade8130d9e0 P 6ce2bc1867f74eb6b113467463a9430a: Log is configured to *not* fsync() on all Append() calls
I20260806 11:39:18.333165 4616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:18.317279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:37871 (local address 127.21.14.193:38467)
0806 11:39:18.318118 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:18.318180 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 11:39:18.319177 (+ 997us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:39:18.320417 (+ 1240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:39:18.320426 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:39:18.320868 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:39:18.321355 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:18.321383 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:18.322750 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:18.322753 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:18.323715 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:18.323727 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:18.324538 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:18.324543 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:18.324755 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:18.325417 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:39:18.325436 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:39:18.327082 (+ 1646us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:39:18.329750 (+ 2668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:39:18.329758 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:39:18.329773 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:39:18.330070 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:39:18.330470 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:39:18.330474 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:39:18.330476 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:39:18.330626 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:39:18.331321 (+ 695us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:39:18.331327 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:39:18.331584 (+ 257us) client_negotiation.cc:769] Sending connection context
0806 11:39:18.331775 (+ 191us) client_negotiation.cc:240] Negotiation successful
0806 11:39:18.332167 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":615,"thread_start_us":145,"threads_started":1}
I20260806 11:39:18.333148 4615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:18.317242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:38467 (local address 127.21.14.254:37871)
0806 11:39:18.317795 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:18.317816 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:18.318959 (+ 1143us) server_negotiation.cc:407] Connection header received
0806 11:39:18.319644 (+ 685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:18.319655 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:18.319984 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:18.320343 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:39:18.321564 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:18.322573 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:18.323874 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:18.324365 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:18.324561 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:18.327310 (+ 2749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:39:18.327317 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:39:18.327331 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:39:18.327363 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:39:18.329599 (+ 2236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:18.330222 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:18.330228 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:18.330232 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:18.330365 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:18.330740 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:18.330743 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:18.330744 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:18.331192 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:39:18.331338 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:18.331914 (+ 576us) server_negotiation.cc:299] Negotiation successful
0806 11:39:18.332194 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":140,"threads_started":1}
I20260806 11:39:18.336346 4613 heartbeater.cc:347] Connected to a master server at 127.21.14.254:37871
I20260806 11:39:18.336627 4613 heartbeater.cc:464] Registering TS with master...
I20260806 11:39:18.337153 4613 heartbeater.cc:511] Master 127.21.14.254:37871 requested a full tablet report, sending...
I20260806 11:39:18.338009 4501 tablet_bootstrap.cc:492] T 314feff7438a46ee927eeade8130d9e0 P 6ce2bc1867f74eb6b113467463a9430a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 11:39:18.338546 4501 tablet_bootstrap.cc:492] T 314feff7438a46ee927eeade8130d9e0 P 6ce2bc1867f74eb6b113467463a9430a: Bootstrap complete.
I20260806 11:39:18.339892 4415 ts_manager.cc:194] Registered new tserver with Master: 6ce2bc1867f74eb6b113467463a9430a (127.21.14.193:43653)
I20260806 11:39:18.340260 4501 ts_tablet_manager.cc:1397] T 314feff7438a46ee927eeade8130d9e0 P 6ce2bc1867f74eb6b113467463a9430a: Time spent bootstrapping tablet: real 0.038s user 0.020s sys 0.014s
I20260806 11:39:18.343533 4415 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.14.193@KRBTEST.COM'} at 127.21.14.193:38467
I20260806 11:39:18.343439 4501 raft_consensus.cc:348] T 314feff7438a46ee927eeade8130d9e0 P 6ce2bc1867f74eb6b113467463a9430a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32e458de791a4f01ac55419385005fa0" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37571 } } peers { permanent_uuid: "f4fc782bf02c45ea9f7ba7a86862f929" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 45493 } } peers { permanent_uuid: "6ce2bc1867f74eb6b113467463a9430a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 43653 } }
I20260806 11:39:18.344107 4501 raft_consensus.cc:729] T 314feff7438a46ee927eeade8130d9e0 P 6ce2bc1867f74eb6b113467463a9430a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ce2bc1867f74eb6b113467463a9430a, State: Initialized, Role: FOLLOWER
I20260806 11:39:18.344696 4501 consensus_queue.cc:260] T 314feff7438a46ee927eeade8130d9e0 P 6ce2bc1867f74eb6b113467463a9430a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32e458de791a4f01ac55419385005fa0" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37571 } } peers { permanent_uuid: "f4fc782bf02c45ea9f7ba7a86862f929" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 45493 } } peers { permanent_uuid: "6ce2bc1867f74eb6b113467463a9430a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 43653 } }
I20260806 11:39:18.345814 4501 ts_tablet_manager.cc:1428] T 314feff7438a46ee927eeade8130d9e0 P 6ce2bc1867f74eb6b113467463a9430a: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260806 11:39:18.346169 4613 heartbeater.cc:503] Master 127.21.14.254:37871 was elected leader, sending a full tablet report...
I20260806 11:39:18.351259 4620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:18.343077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52712 (local address 127.21.14.193:43653)
0806 11:39:18.344580 (+ 1503us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:18.344600 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:18.346502 (+ 1902us) server_negotiation.cc:407] Connection header received
0806 11:39:18.346715 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:18.346720 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:18.346971 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:18.347148 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:39:18.347450 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:18.348380 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:18.349721 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:18.350513 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:18.350625 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:18.350784 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:18.350991 (+ 207us) server_negotiation.cc:299] Negotiation successful
0806 11:39:18.351102 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":970,"thread_start_us":111,"threads_started":1}
I20260806 11:39:18.351259 4621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:18.342807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.193:43653 (local address 127.0.0.1:52712)
0806 11:39:18.346323 (+ 3516us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:18.346381 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 11:39:18.346553 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:39:18.347156 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:39:18.347162 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:39:18.347188 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:39:18.347328 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:18.347340 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:18.348550 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:18.348554 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:18.349601 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:18.349612 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:18.350657 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:18.350660 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:18.350757 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:18.350791 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 11:39:18.350944 (+ 153us) client_negotiation.cc:240] Negotiation successful
0806 11:39:18.351103 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1421,"thread_start_us":588,"threads_started":1}
I20260806 11:39:18.352387 4546 tablet_service.cc:1514] Processing DeleteTablet for tablet 314feff7438a46ee927eeade8130d9e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:39:17 UTC) from {username='kudu', principal='kudu/127.21.14.254@KRBTEST.COM'} at 127.0.0.1:52712
I20260806 11:39:18.353271 4623 tablet_replica.cc:330] stopping tablet replica
I20260806 11:39:18.353598 4623 raft_consensus.cc:2229] T 314feff7438a46ee927eeade8130d9e0 P 6ce2bc1867f74eb6b113467463a9430a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 11:39:18.353864 4623 raft_consensus.cc:2258] T 314feff7438a46ee927eeade8130d9e0 P 6ce2bc1867f74eb6b113467463a9430a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 11:39:18.354964 4623 ts_tablet_manager.cc:1905] T 314feff7438a46ee927eeade8130d9e0 P 6ce2bc1867f74eb6b113467463a9430a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 11:39:18.359115 4623 ts_tablet_manager.cc:1918] T 314feff7438a46ee927eeade8130d9e0 P 6ce2bc1867f74eb6b113467463a9430a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 11:39:18.359222 4623 log.cc:1196] T 314feff7438a46ee927eeade8130d9e0 P 6ce2bc1867f74eb6b113467463a9430a: Deleting WAL directory at /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-0/wal/wals/314feff7438a46ee927eeade8130d9e0
I20260806 11:39:18.359508 4623 ts_tablet_manager.cc:1939] T 314feff7438a46ee927eeade8130d9e0 P 6ce2bc1867f74eb6b113467463a9430a: Deleting consensus metadata
I20260806 11:39:18.361418 4397 catalog_manager.cc:4841] TS 6ce2bc1867f74eb6b113467463a9430a (127.21.14.193:43653): tablet 314feff7438a46ee927eeade8130d9e0 (table test-table [id=0188b6877aca49169233cd6df4a59e67]) successfully deleted
W20260806 11:39:18.454100 4618 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:39:18.454440 4618 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:39:18.454550 4618 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:39:18.458978 4618 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:39:18.459076 4618 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:39:18.459172 4618 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.194
I20260806 11:39:18.464907 4618 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:45535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab
--principal=kudu/127.21.14.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.194:37571
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.14.194
--webserver_port=42217
--tserver_master_addrs=127.21.14.254:37871
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4618
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.194
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:39:18.466131 4618 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:39:18.467298 4618 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:39:18.477840 4628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:39:18.477833 4629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:39:18.477833 4631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:39:18.478271 4618 server_base.cc:1034] running on GCE node
I20260806 11:39:18.478773 4618 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:39:18.479519 4618 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:39:18.480693 4618 hybrid_clock.cc:648] HybridClock initialized: now 1786016358480655 us; error 58 us; skew 500 ppm
Aug 06 11:39:18 dist-test-slave-7c35 krb5kdc[3793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016358, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:39:18.483592 4618 init.cc:377] Logged in from keytab as kudu/127.21.14.194@KRBTEST.COM (short username kudu)
I20260806 11:39:18.484491 4618 webserver.cc:460] Webserver started at http://127.21.14.194:42217/ using document root <none> and password file <none>
I20260806 11:39:18.485052 4618 fs_manager.cc:359] Metadata directory not provided
I20260806 11:39:18.485126 4618 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:39:18.489161 4618 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 11:39:18.491446 4638 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:39:18.492887 4618 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 11:39:18.493005 4618 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-1/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-1/wal
uuid: "32e458de791a4f01ac55419385005fa0"
format_stamp: "Formatted at 2026-08-06 11:39:17 on dist-test-slave-7c35"
server_key: "938466b98f9733c6a445968b4733bdb9"
server_key_iv: "a7e538923c63b93b4c6d1effc5fb4cc6"
server_key_version: "encryptionkey@0"
I20260806 11:39:18.493592 4618 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:39:18.511759 4618 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:39:18.515362 4618 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:39:18.515573 4618 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:39:18.516196 4618 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:39:18.517189 4618 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:39:18.517241 4618 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:39:18.517284 4618 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:39:18.517300 4618 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:39:18.526613 4618 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.194:37571
I20260806 11:39:18.526676 4768 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.194:37571 every 8 connection(s)
I20260806 11:39:18.527812 4618 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
I20260806 11:39:18.528731 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 4618
I20260806 11:39:18.529268 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.195:45493
--local_ip_for_outbound_sockets=127.21.14.195
--tserver_master_addrs=127.21.14.254:37871
--webserver_port=39387
--webserver_interface=127.21.14.195
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab
--principal=kudu/127.21.14.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:45535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 11:39:18 dist-test-slave-7c35 krb5kdc[3793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016358, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.194@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:39:18.544744 4615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:18.530237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:51701 (local address 127.21.14.254:37871)
0806 11:39:18.530398 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:18.530402 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:18.531776 (+ 1374us) server_negotiation.cc:407] Connection header received
0806 11:39:18.533720 (+ 1944us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:18.533724 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:18.533807 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:18.533904 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:39:18.535335 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:18.535876 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:18.537098 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:18.537346 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:18.537463 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:18.540585 (+ 3122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:39:18.540589 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:39:18.540592 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:39:18.540620 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:39:18.542341 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:18.542993 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:18.542996 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:18.542997 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:18.543081 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:18.543455 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:18.543458 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:18.543460 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:18.543695 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:39:18.543807 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:18.544394 (+ 587us) server_negotiation.cc:299] Negotiation successful
0806 11:39:18.544562 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 11:39:18.545785 4772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:18.530779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:37871 (local address 127.21.14.194:51701)
0806 11:39:18.531546 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:18.531629 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0806 11:39:18.533437 (+ 1808us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:39:18.534129 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:39:18.534137 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:39:18.534560 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:39:18.535155 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:18.535171 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:18.536031 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:18.536034 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:18.536962 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:18.536973 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:18.537526 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:18.537531 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:18.537719 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:18.538548 (+ 829us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:39:18.538568 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:39:18.540359 (+ 1791us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:39:18.542503 (+ 2144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:39:18.542510 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:39:18.542529 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:39:18.542841 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:39:18.543178 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:39:18.543181 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:39:18.543183 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:39:18.543350 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:39:18.543811 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:39:18.543821 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:39:18.544130 (+ 309us) client_negotiation.cc:769] Sending connection context
0806 11:39:18.544338 (+ 208us) client_negotiation.cc:240] Negotiation successful
0806 11:39:18.544725 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":159,"threads_started":1}
I20260806 11:39:18.547363 4769 heartbeater.cc:347] Connected to a master server at 127.21.14.254:37871
I20260806 11:39:18.547696 4769 heartbeater.cc:464] Registering TS with master...
I20260806 11:39:18.548370 4769 heartbeater.cc:511] Master 127.21.14.254:37871 requested a full tablet report, sending...
I20260806 11:39:18.549660 4415 ts_manager.cc:194] Registered new tserver with Master: 32e458de791a4f01ac55419385005fa0 (127.21.14.194:37571)
I20260806 11:39:18.550318 4415 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.14.194@KRBTEST.COM'} at 127.21.14.194:51701
W20260806 11:39:18.654397 4773 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:39:18.654764 4773 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:39:18.654898 4773 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:39:18.659449 4773 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:39:18.659544 4773 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:39:18.659636 4773 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.195
I20260806 11:39:18.665627 4773 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:45535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab
--principal=kudu/127.21.14.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.195:45493
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.14.195
--webserver_port=39387
--tserver_master_addrs=127.21.14.254:37871
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4773
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.195
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:39:18.666934 4773 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:39:18.668131 4773 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:39:18.678911 4779 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:39:18.678872 4778 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:39:18.678872 4781 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:39:18.679322 4773 server_base.cc:1034] running on GCE node
I20260806 11:39:18.679787 4773 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:39:18.680560 4773 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:39:18.681771 4773 hybrid_clock.cc:648] HybridClock initialized: now 1786016358681735 us; error 54 us; skew 500 ppm
Aug 06 11:39:18 dist-test-slave-7c35 krb5kdc[3793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016358, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:39:18.684828 4773 init.cc:377] Logged in from keytab as kudu/127.21.14.195@KRBTEST.COM (short username kudu)
I20260806 11:39:18.685848 4773 webserver.cc:460] Webserver started at http://127.21.14.195:39387/ using document root <none> and password file <none>
I20260806 11:39:18.686472 4773 fs_manager.cc:359] Metadata directory not provided
I20260806 11:39:18.686599 4773 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:39:18.691160 4773 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 11:39:18.693835 4789 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:39:18.695593 4773 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 11:39:18.695780 4773 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-2/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-2/wal
uuid: "f4fc782bf02c45ea9f7ba7a86862f929"
format_stamp: "Formatted at 2026-08-06 11:39:17 on dist-test-slave-7c35"
server_key: "559653b6b7a910b2ca05056057139afc"
server_key_iv: "0b8e20d7bf224591b44bde6d71b8ab90"
server_key_version: "encryptionkey@0"
I20260806 11:39:18.696460 4773 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:39:18.709089 4773 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:39:18.713057 4773 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:39:18.713313 4773 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:39:18.714114 4773 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:39:18.715265 4773 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:39:18.715358 4773 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:39:18.715438 4773 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:39:18.715488 4773 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:39:18.726526 4773 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.195:45493
I20260806 11:39:18.726581 4912 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.195:45493 every 8 connection(s)
I20260806 11:39:18.727728 4773 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
I20260806 11:39:18.728863 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 4773
I20260806 11:39:18.739341 4615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:18.729721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36468 (local address 127.21.14.254:37871)
0806 11:39:18.729889 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:18.729897 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:18.730264 (+ 367us) server_negotiation.cc:407] Connection header received
0806 11:39:18.730422 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:18.730425 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:18.730481 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:18.730571 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:39:18.731471 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:18.732020 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:18.733067 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:18.733307 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:18.733414 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:18.735224 (+ 1810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:39:18.735228 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:39:18.735230 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:39:18.735258 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:39:18.737175 (+ 1917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:18.737792 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:18.737796 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:18.737797 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:18.737901 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:18.738185 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:18.738189 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:18.738191 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:18.738484 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:39:18.738789 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:18.739035 (+ 246us) server_negotiation.cc:299] Negotiation successful
0806 11:39:18.739165 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
Aug 06 11:39:18 dist-test-slave-7c35 krb5kdc[3793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016358, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.195@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:39:18.746151 4916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:18.729815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:33407 (local address 127.21.14.254:37871)
0806 11:39:18.730057 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:18.730062 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:18.731154 (+ 1092us) server_negotiation.cc:407] Connection header received
0806 11:39:18.732702 (+ 1548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:18.732705 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:18.732757 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:18.732853 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:39:18.734360 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:18.735252 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:18.737142 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:18.737504 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:18.737632 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:18.741456 (+ 3824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:39:18.741462 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:39:18.741464 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:39:18.741510 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:39:18.743596 (+ 2086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:18.744335 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:18.744340 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:18.744341 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:18.744509 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:18.744965 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:18.744968 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:18.744969 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:18.745196 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:39:18.745285 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:18.745860 (+ 575us) server_negotiation.cc:299] Negotiation successful
0806 11:39:18.746005 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":98,"threads_started":1}
I20260806 11:39:18.747239 4917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:18.730436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:37871 (local address 127.21.14.195:33407)
0806 11:39:18.730978 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:18.731036 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 11:39:18.732472 (+ 1436us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:39:18.733047 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:39:18.733055 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:39:18.733497 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:39:18.734195 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:18.734215 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:18.735396 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:18.735400 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:18.736965 (+ 1565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:18.736982 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:18.737730 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:18.737737 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:18.737998 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:18.738698 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:39:18.738743 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:39:18.741055 (+ 2312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:39:18.743776 (+ 2721us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:39:18.743783 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:39:18.743800 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:39:18.744168 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:39:18.744642 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:39:18.744646 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:39:18.744647 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:39:18.744844 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:39:18.745308 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:39:18.745315 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:39:18.745625 (+ 310us) client_negotiation.cc:769] Sending connection context
0806 11:39:18.745810 (+ 185us) client_negotiation.cc:240] Negotiation successful
0806 11:39:18.746128 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":146,"threads_started":1}
I20260806 11:39:18.748668 4913 heartbeater.cc:347] Connected to a master server at 127.21.14.254:37871
I20260806 11:39:18.748968 4913 heartbeater.cc:464] Registering TS with master...
I20260806 11:39:18.749573 4913 heartbeater.cc:511] Master 127.21.14.254:37871 requested a full tablet report, sending...
I20260806 11:39:18.750825 4415 ts_manager.cc:194] Registered new tserver with Master: f4fc782bf02c45ea9f7ba7a86862f929 (127.21.14.195:45493)
I20260806 11:39:18.751528 4415 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.14.195@KRBTEST.COM'} at 127.21.14.195:33407
I20260806 11:39:18.753989 21563 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 11:39:18.758759 4916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:18.754590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36482 (local address 127.21.14.254:37871)
0806 11:39:18.754738 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:18.754743 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:18.754960 (+ 217us) server_negotiation.cc:407] Connection header received
0806 11:39:18.755114 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:18.755116 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:18.755224 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:18.755312 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 11:39:18.755566 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:18.756101 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:18.756983 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:18.757245 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:18.757338 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:18.757775 (+ 437us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 11:39:18.758255 (+ 480us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 11:39:18.758351 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:18.758455 (+ 104us) server_negotiation.cc:299] Negotiation successful
0806 11:39:18.758519 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 11:39:18.759562 4415 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:36482:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 11:39:18.763074 4415 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 11:39:18.772476 4546 tablet_service.cc:1467] Processing CreateTablet for tablet 6a8e14e2573b4ddc98d0fe9fb885fcf8 (DEFAULT_TABLE table=test-table [id=c81125daf8124c309a6bd41fd7195d41]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:39:18.773216 4546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a8e14e2573b4ddc98d0fe9fb885fcf8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:39:18.776156 4621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:18.770733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.195:45493 (local address 127.0.0.1:54172)
0806 11:39:18.770881 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:18.770899 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 11:39:18.770993 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:39:18.772019 (+ 1026us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:39:18.772022 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:39:18.772042 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:39:18.772158 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:18.772164 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:18.773531 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:18.773537 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:18.774442 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:18.774452 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:18.775693 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:18.775697 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:18.775785 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:18.775828 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 11:39:18.775909 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 11:39:18.776011 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260806 11:39:18.776180 4923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:18.770994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54172 (local address 127.21.14.195:45493)
0806 11:39:18.771462 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:18.771476 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:18.771505 (+ 29us) server_negotiation.cc:407] Connection header received
0806 11:39:18.771604 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:18.771611 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:18.771867 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:18.772063 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:39:18.772266 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:18.773373 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:18.774572 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:18.775567 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:18.775691 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:18.775857 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:18.775949 (+ 92us) server_negotiation.cc:299] Negotiation successful
0806 11:39:18.776048 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":199,"threads_started":1}
I20260806 11:39:18.776527 4922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:18.771068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.194:37571 (local address 127.0.0.1:60180)
0806 11:39:18.771338 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:18.771353 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 11:39:18.771447 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:39:18.772397 (+ 950us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:39:18.772400 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:39:18.772418 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:39:18.772519 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:18.772525 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:18.773667 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:18.773671 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:18.774915 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:18.774926 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:18.776140 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:18.776144 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:18.776236 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:18.776261 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 11:39:18.776320 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 11:39:18.776401 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":114,"threads_started":1}
I20260806 11:39:18.776683 4501 tablet_bootstrap.cc:492] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 6ce2bc1867f74eb6b113467463a9430a: Bootstrap starting.
I20260806 11:39:18.777314 4924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:18.771540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60180 (local address 127.21.14.194:37571)
0806 11:39:18.771854 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:18.771866 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:18.771888 (+ 22us) server_negotiation.cc:407] Connection header received
0806 11:39:18.772043 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:18.772049 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:18.772255 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:18.772424 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:39:18.772604 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:18.773507 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:18.775087 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:18.775977 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:18.776820 (+ 843us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:18.776984 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:18.777040 (+ 56us) server_negotiation.cc:299] Negotiation successful
0806 11:39:18.777157 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":124,"threads_started":1}
I20260806 11:39:18.777715 4501 tablet_bootstrap.cc:654] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 6ce2bc1867f74eb6b113467463a9430a: Neither blocks nor log segments found. Creating new log.
I20260806 11:39:18.778939 4501 tablet_bootstrap.cc:492] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 6ce2bc1867f74eb6b113467463a9430a: No bootstrap required, opened a new log
I20260806 11:39:18.779040 4501 ts_tablet_manager.cc:1397] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 6ce2bc1867f74eb6b113467463a9430a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.001s
I20260806 11:39:18.779424 4501 raft_consensus.cc:348] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 6ce2bc1867f74eb6b113467463a9430a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4fc782bf02c45ea9f7ba7a86862f929" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 45493 } } peers { permanent_uuid: "32e458de791a4f01ac55419385005fa0" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37571 } } peers { permanent_uuid: "6ce2bc1867f74eb6b113467463a9430a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 43653 } }
I20260806 11:39:18.779520 4501 raft_consensus.cc:374] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 6ce2bc1867f74eb6b113467463a9430a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:39:18.779548 4501 raft_consensus.cc:729] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 6ce2bc1867f74eb6b113467463a9430a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ce2bc1867f74eb6b113467463a9430a, State: Initialized, Role: FOLLOWER
I20260806 11:39:18.779696 4501 consensus_queue.cc:260] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 6ce2bc1867f74eb6b113467463a9430a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4fc782bf02c45ea9f7ba7a86862f929" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 45493 } } peers { permanent_uuid: "32e458de791a4f01ac55419385005fa0" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37571 } } peers { permanent_uuid: "6ce2bc1867f74eb6b113467463a9430a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 43653 } }
I20260806 11:39:18.779973 4501 ts_tablet_manager.cc:1428] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 6ce2bc1867f74eb6b113467463a9430a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 11:39:18.779894 4842 tablet_service.cc:1467] Processing CreateTablet for tablet 6a8e14e2573b4ddc98d0fe9fb885fcf8 (DEFAULT_TABLE table=test-table [id=c81125daf8124c309a6bd41fd7195d41]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:39:18.780323 4692 tablet_service.cc:1467] Processing CreateTablet for tablet 6a8e14e2573b4ddc98d0fe9fb885fcf8 (DEFAULT_TABLE table=test-table [id=c81125daf8124c309a6bd41fd7195d41]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:39:18.781170 4842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a8e14e2573b4ddc98d0fe9fb885fcf8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:39:18.781337 4692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a8e14e2573b4ddc98d0fe9fb885fcf8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:39:18.790355 4927 tablet_bootstrap.cc:492] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 32e458de791a4f01ac55419385005fa0: Bootstrap starting.
I20260806 11:39:18.791042 4926 tablet_bootstrap.cc:492] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P f4fc782bf02c45ea9f7ba7a86862f929: Bootstrap starting.
I20260806 11:39:18.793147 4926 tablet_bootstrap.cc:654] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P f4fc782bf02c45ea9f7ba7a86862f929: Neither blocks nor log segments found. Creating new log.
I20260806 11:39:18.793664 4927 tablet_bootstrap.cc:654] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 32e458de791a4f01ac55419385005fa0: Neither blocks nor log segments found. Creating new log.
I20260806 11:39:18.794616 4927 log.cc:824] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 32e458de791a4f01ac55419385005fa0: Log is configured to *not* fsync() on all Append() calls
I20260806 11:39:18.794616 4926 log.cc:824] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P f4fc782bf02c45ea9f7ba7a86862f929: Log is configured to *not* fsync() on all Append() calls
I20260806 11:39:18.796787 4926 tablet_bootstrap.cc:492] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P f4fc782bf02c45ea9f7ba7a86862f929: No bootstrap required, opened a new log
I20260806 11:39:18.796787 4927 tablet_bootstrap.cc:492] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 32e458de791a4f01ac55419385005fa0: No bootstrap required, opened a new log
I20260806 11:39:18.797032 4927 ts_tablet_manager.cc:1397] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 32e458de791a4f01ac55419385005fa0: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 11:39:18.797046 4926 ts_tablet_manager.cc:1397] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P f4fc782bf02c45ea9f7ba7a86862f929: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 11:39:18.800653 4927 raft_consensus.cc:348] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 32e458de791a4f01ac55419385005fa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4fc782bf02c45ea9f7ba7a86862f929" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 45493 } } peers { permanent_uuid: "32e458de791a4f01ac55419385005fa0" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37571 } } peers { permanent_uuid: "6ce2bc1867f74eb6b113467463a9430a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 43653 } }
I20260806 11:39:18.800653 4926 raft_consensus.cc:348] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P f4fc782bf02c45ea9f7ba7a86862f929 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4fc782bf02c45ea9f7ba7a86862f929" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 45493 } } peers { permanent_uuid: "32e458de791a4f01ac55419385005fa0" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37571 } } peers { permanent_uuid: "6ce2bc1867f74eb6b113467463a9430a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 43653 } }
I20260806 11:39:18.800915 4927 raft_consensus.cc:374] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 32e458de791a4f01ac55419385005fa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:39:18.800936 4926 raft_consensus.cc:374] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P f4fc782bf02c45ea9f7ba7a86862f929 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:39:18.800982 4927 raft_consensus.cc:729] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 32e458de791a4f01ac55419385005fa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32e458de791a4f01ac55419385005fa0, State: Initialized, Role: FOLLOWER
I20260806 11:39:18.801002 4926 raft_consensus.cc:729] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P f4fc782bf02c45ea9f7ba7a86862f929 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4fc782bf02c45ea9f7ba7a86862f929, State: Initialized, Role: FOLLOWER
I20260806 11:39:18.801476 4926 consensus_queue.cc:260] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P f4fc782bf02c45ea9f7ba7a86862f929 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4fc782bf02c45ea9f7ba7a86862f929" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 45493 } } peers { permanent_uuid: "32e458de791a4f01ac55419385005fa0" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37571 } } peers { permanent_uuid: "6ce2bc1867f74eb6b113467463a9430a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 43653 } }
I20260806 11:39:18.801476 4927 consensus_queue.cc:260] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 32e458de791a4f01ac55419385005fa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4fc782bf02c45ea9f7ba7a86862f929" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 45493 } } peers { permanent_uuid: "32e458de791a4f01ac55419385005fa0" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37571 } } peers { permanent_uuid: "6ce2bc1867f74eb6b113467463a9430a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 43653 } }
I20260806 11:39:18.802253 4913 heartbeater.cc:503] Master 127.21.14.254:37871 was elected leader, sending a full tablet report...
I20260806 11:39:18.802479 4769 heartbeater.cc:503] Master 127.21.14.254:37871 was elected leader, sending a full tablet report...
I20260806 11:39:18.802616 4926 ts_tablet_manager.cc:1428] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P f4fc782bf02c45ea9f7ba7a86862f929: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 11:39:18.802614 4927 ts_tablet_manager.cc:1428] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 32e458de791a4f01ac55419385005fa0: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260806 11:39:18.816735 4614 tablet.cc:2369] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 6ce2bc1867f74eb6b113467463a9430a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 11:39:18.880875 4931 raft_consensus.cc:482] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 32e458de791a4f01ac55419385005fa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 11:39:18.881095 4931 raft_consensus.cc:504] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 32e458de791a4f01ac55419385005fa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4fc782bf02c45ea9f7ba7a86862f929" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 45493 } } peers { permanent_uuid: "32e458de791a4f01ac55419385005fa0" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37571 } } peers { permanent_uuid: "6ce2bc1867f74eb6b113467463a9430a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 43653 } }
I20260806 11:39:18.882421 4931 leader_election.cc:288] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 32e458de791a4f01ac55419385005fa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f4fc782bf02c45ea9f7ba7a86862f929 (127.21.14.195:45493), 6ce2bc1867f74eb6b113467463a9430a (127.21.14.193:43653)
I20260806 11:39:18.886929 4923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:18.882787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:44317 (local address 127.21.14.195:45493)
0806 11:39:18.882991 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:18.882997 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:18.883019 (+ 22us) server_negotiation.cc:407] Connection header received
0806 11:39:18.883192 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:18.883196 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:18.883256 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:18.883364 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:39:18.883717 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:18.884399 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:18.885682 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:18.886215 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:18.886337 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:18.886386 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:18.886624 (+ 238us) server_negotiation.cc:299] Negotiation successful
0806 11:39:18.886754 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 11:39:18.886911 4772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:18.882716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.195:45493 (local address 127.21.14.194:44317)
0806 11:39:18.882900 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:18.882920 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 11:39:18.883045 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:39:18.883391 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:39:18.883394 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:39:18.883416 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:39:18.883583 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:18.883592 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:18.884515 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:18.884519 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:18.885519 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:18.885530 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:18.886337 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:18.886341 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:18.886458 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:18.886486 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 11:39:18.886604 (+ 118us) client_negotiation.cc:240] Negotiation successful
0806 11:39:18.886703 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52,"mutex_wait_us":47}
I20260806 11:39:18.887223 4933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:18.882720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.193:43653 (local address 127.21.14.194:60129)
0806 11:39:18.883387 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:18.883403 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 11:39:18.883516 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:39:18.883811 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:39:18.883814 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:39:18.883856 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:39:18.884003 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:18.884012 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:18.884866 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:18.884869 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:18.885854 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:18.885866 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:18.886721 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:18.886726 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:18.886828 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:18.886892 (+ 64us) client_negotiation.cc:769] Sending connection context
0806 11:39:18.886984 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 11:39:18.887059 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"mutex_wait_us":195,"thread_start_us":155,"threads_started":1}
I20260806 11:39:18.887265 4932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:18.882982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:60129 (local address 127.21.14.193:43653)
0806 11:39:18.883300 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:18.883304 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:18.883491 (+ 187us) server_negotiation.cc:407] Connection header received
0806 11:39:18.883622 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:18.883627 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:18.883693 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:18.883779 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:39:18.884131 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:18.884722 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:18.885985 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:18.886579 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:18.886697 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:18.886745 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:18.887009 (+ 264us) server_negotiation.cc:299] Negotiation successful
0806 11:39:18.887112 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":239,"thread_start_us":104,"threads_started":1}
I20260806 11:39:18.887789 4863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a8e14e2573b4ddc98d0fe9fb885fcf8" candidate_uuid: "32e458de791a4f01ac55419385005fa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4fc782bf02c45ea9f7ba7a86862f929" is_pre_election: true
I20260806 11:39:18.887907 4566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a8e14e2573b4ddc98d0fe9fb885fcf8" candidate_uuid: "32e458de791a4f01ac55419385005fa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ce2bc1867f74eb6b113467463a9430a" is_pre_election: true
I20260806 11:39:18.888182 4863 raft_consensus.cc:2454] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P f4fc782bf02c45ea9f7ba7a86862f929 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32e458de791a4f01ac55419385005fa0 in term 0.
I20260806 11:39:18.888250 4566 raft_consensus.cc:2454] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 6ce2bc1867f74eb6b113467463a9430a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32e458de791a4f01ac55419385005fa0 in term 0.
I20260806 11:39:18.888724 4640 leader_election.cc:302] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 32e458de791a4f01ac55419385005fa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32e458de791a4f01ac55419385005fa0, f4fc782bf02c45ea9f7ba7a86862f929; no voters:
I20260806 11:39:18.889039 4931 raft_consensus.cc:2781] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 32e458de791a4f01ac55419385005fa0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 11:39:18.889142 4931 raft_consensus.cc:482] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 32e458de791a4f01ac55419385005fa0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 11:39:18.889189 4931 raft_consensus.cc:3037] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 32e458de791a4f01ac55419385005fa0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:39:18.890311 4931 raft_consensus.cc:504] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 32e458de791a4f01ac55419385005fa0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4fc782bf02c45ea9f7ba7a86862f929" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 45493 } } peers { permanent_uuid: "32e458de791a4f01ac55419385005fa0" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37571 } } peers { permanent_uuid: "6ce2bc1867f74eb6b113467463a9430a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 43653 } }
I20260806 11:39:18.890703 4931 leader_election.cc:288] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 32e458de791a4f01ac55419385005fa0 [CANDIDATE]: Term 1 election: Requested vote from peers f4fc782bf02c45ea9f7ba7a86862f929 (127.21.14.195:45493), 6ce2bc1867f74eb6b113467463a9430a (127.21.14.193:43653)
I20260806 11:39:18.891099 4863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a8e14e2573b4ddc98d0fe9fb885fcf8" candidate_uuid: "32e458de791a4f01ac55419385005fa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4fc782bf02c45ea9f7ba7a86862f929"
I20260806 11:39:18.891139 4566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a8e14e2573b4ddc98d0fe9fb885fcf8" candidate_uuid: "32e458de791a4f01ac55419385005fa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ce2bc1867f74eb6b113467463a9430a"
I20260806 11:39:18.891270 4863 raft_consensus.cc:3037] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P f4fc782bf02c45ea9f7ba7a86862f929 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:39:18.891283 4566 raft_consensus.cc:3037] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 6ce2bc1867f74eb6b113467463a9430a [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:39:18.892408 4863 raft_consensus.cc:2454] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P f4fc782bf02c45ea9f7ba7a86862f929 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32e458de791a4f01ac55419385005fa0 in term 1.
I20260806 11:39:18.892413 4566 raft_consensus.cc:2454] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 6ce2bc1867f74eb6b113467463a9430a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32e458de791a4f01ac55419385005fa0 in term 1.
I20260806 11:39:18.892884 4639 leader_election.cc:302] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 32e458de791a4f01ac55419385005fa0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32e458de791a4f01ac55419385005fa0, 6ce2bc1867f74eb6b113467463a9430a; no voters:
I20260806 11:39:18.893208 4931 raft_consensus.cc:2781] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 32e458de791a4f01ac55419385005fa0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:39:18.893472 4931 raft_consensus.cc:686] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 32e458de791a4f01ac55419385005fa0 [term 1 LEADER]: Becoming Leader. State: Replica: 32e458de791a4f01ac55419385005fa0, State: Running, Role: LEADER
I20260806 11:39:18.893895 4931 consensus_queue.cc:237] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 32e458de791a4f01ac55419385005fa0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4fc782bf02c45ea9f7ba7a86862f929" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 45493 } } peers { permanent_uuid: "32e458de791a4f01ac55419385005fa0" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37571 } } peers { permanent_uuid: "6ce2bc1867f74eb6b113467463a9430a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 43653 } }
I20260806 11:39:18.897570 4415 catalog_manager.cc:5495] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 32e458de791a4f01ac55419385005fa0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 32e458de791a4f01ac55419385005fa0 (127.21.14.194). New cstate: current_term: 1 leader_uuid: "32e458de791a4f01ac55419385005fa0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4fc782bf02c45ea9f7ba7a86862f929" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 45493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32e458de791a4f01ac55419385005fa0" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37571 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ce2bc1867f74eb6b113467463a9430a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 43653 } health_report { overall_health: UNKNOWN } } }
I20260806 11:39:18.938974 4924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:18.935177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60194 (local address 127.21.14.194:37571)
0806 11:39:18.935341 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:18.935346 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:18.935365 (+ 19us) server_negotiation.cc:407] Connection header received
0806 11:39:18.935447 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:18.935449 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:18.935536 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:18.935657 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 11:39:18.935914 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:18.936433 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:18.937295 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:18.937517 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:18.937624 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:18.938007 (+ 383us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 11:39:18.938512 (+ 505us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 11:39:18.938647 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:18.938689 (+ 42us) server_negotiation.cc:299] Negotiation successful
0806 11:39:18.938755 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 11:39:18.945552 4566 raft_consensus.cc:1261] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 6ce2bc1867f74eb6b113467463a9430a [term 1 FOLLOWER]: Refusing update from remote peer 32e458de791a4f01ac55419385005fa0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 11:39:18.945557 4863 raft_consensus.cc:1261] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P f4fc782bf02c45ea9f7ba7a86862f929 [term 1 FOLLOWER]: Refusing update from remote peer 32e458de791a4f01ac55419385005fa0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 11:39:18.946341 4934 consensus_queue.cc:1035] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 32e458de791a4f01ac55419385005fa0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4fc782bf02c45ea9f7ba7a86862f929" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 45493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 11:39:18.946544 4931 consensus_queue.cc:1035] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 32e458de791a4f01ac55419385005fa0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ce2bc1867f74eb6b113467463a9430a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 43653 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 11:39:18.963913 4415 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:36482:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 11:39:18.964092 4415 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:36482:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 11:39:18.966706 4415 catalog_manager.cc:5782] T 00000000000000000000000000000000 P beea7743cac049cfb113cc825011d29e: Sending DeleteTablet for 3 replicas of tablet 6a8e14e2573b4ddc98d0fe9fb885fcf8
I20260806 11:39:18.967640 4842 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a8e14e2573b4ddc98d0fe9fb885fcf8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:39:18 UTC) from {username='kudu', principal='kudu/127.21.14.254@KRBTEST.COM'} at 127.0.0.1:54172
I20260806 11:39:18.967800 4546 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a8e14e2573b4ddc98d0fe9fb885fcf8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:39:18 UTC) from {username='kudu', principal='kudu/127.21.14.254@KRBTEST.COM'} at 127.0.0.1:52712
I20260806 11:39:18.967851 4692 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a8e14e2573b4ddc98d0fe9fb885fcf8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:39:18 UTC) from {username='kudu', principal='kudu/127.21.14.254@KRBTEST.COM'} at 127.0.0.1:60180
I20260806 11:39:18.968274 4947 tablet_replica.cc:330] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P f4fc782bf02c45ea9f7ba7a86862f929: stopping tablet replica
I20260806 11:39:18.968416 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 4457
I20260806 11:39:18.968595 4946 tablet_replica.cc:330] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 6ce2bc1867f74eb6b113467463a9430a: stopping tablet replica
I20260806 11:39:18.970904 4948 tablet_replica.cc:330] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 32e458de791a4f01ac55419385005fa0: stopping tablet replica
I20260806 11:39:18.971412 4948 raft_consensus.cc:2229] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 32e458de791a4f01ac55419385005fa0 [term 1 LEADER]: Raft consensus shutting down.
I20260806 11:39:18.971531 4947 raft_consensus.cc:2229] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P f4fc782bf02c45ea9f7ba7a86862f929 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 11:39:18.971953 4947 raft_consensus.cc:2258] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P f4fc782bf02c45ea9f7ba7a86862f929 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 11:39:18.972100 4948 raft_consensus.cc:2258] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 32e458de791a4f01ac55419385005fa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 11:39:18.973769 4948 ts_tablet_manager.cc:1905] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 32e458de791a4f01ac55419385005fa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 11:39:18.977053 4948 ts_tablet_manager.cc:1918] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 32e458de791a4f01ac55419385005fa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 11:39:18.977200 4948 log.cc:1196] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 32e458de791a4f01ac55419385005fa0: Deleting WAL directory at /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-1/wal/wals/6a8e14e2573b4ddc98d0fe9fb885fcf8
I20260806 11:39:18.977687 4948 ts_tablet_manager.cc:1939] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 32e458de791a4f01ac55419385005fa0: Deleting consensus metadata
W20260806 11:39:18.979362 4397 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.14.193:43653 (error 108)
I20260806 11:39:18.979514 4402 catalog_manager.cc:4841] TS 32e458de791a4f01ac55419385005fa0 (127.21.14.194:37571): tablet 6a8e14e2573b4ddc98d0fe9fb885fcf8 (table test-table [id=c81125daf8124c309a6bd41fd7195d41]) successfully deleted
I20260806 11:39:18.980417 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 4618
I20260806 11:39:18.980679 4947 ts_tablet_manager.cc:1905] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P f4fc782bf02c45ea9f7ba7a86862f929: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 11:39:18.984100 4947 ts_tablet_manager.cc:1918] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P f4fc782bf02c45ea9f7ba7a86862f929: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 11:39:18.984267 4947 log.cc:1196] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P f4fc782bf02c45ea9f7ba7a86862f929: Deleting WAL directory at /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-2/wal/wals/6a8e14e2573b4ddc98d0fe9fb885fcf8
I20260806 11:39:18.984673 4947 ts_tablet_manager.cc:1939] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P f4fc782bf02c45ea9f7ba7a86862f929: Deleting consensus metadata
I20260806 11:39:18.985425 4922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:18.985062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.193:43653 (local address 127.0.0.1:52714)
0806 11:39:18.985232 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:18.985326 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.193:43653: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260806 11:39:18.986272 4399 catalog_manager.cc:4841] TS f4fc782bf02c45ea9f7ba7a86862f929 (127.21.14.195:45493): tablet 6a8e14e2573b4ddc98d0fe9fb885fcf8 (table test-table [id=c81125daf8124c309a6bd41fd7195d41]) successfully deleted
W20260806 11:39:18.985719 4397 catalog_manager.cc:4584] TS 6ce2bc1867f74eb6b113467463a9430a (127.21.14.193:43653): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6a8e14e2573b4ddc98d0fe9fb885fcf8: Network error: Client connection negotiation failed: client connection to 127.21.14.193:43653: connect: Connection refused (error 111)
I20260806 11:39:18.989741 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 4773
I20260806 11:39:18.997987 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 4379
I20260806 11:39:19.007761 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:37871
--webserver_interface=127.21.14.254
--webserver_port=38567
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab
--principal=kudu/127.21.14.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:45535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.14.254:37871
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:39:19.129534 4950 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:39:19.129879 4950 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:39:19.129926 4950 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:39:19.134336 4950 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 11:39:19.134438 4950 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:39:19.134459 4950 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:39:19.134476 4950 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 11:39:19.134498 4950 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 11:39:19.140429 4950 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:45535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.14.254:37871
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab
--principal=kudu/127.21.14.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:37871
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.14.254
--webserver_port=38567
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4950
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:39:19.141737 4950 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:39:19.142944 4950 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:39:19.152777 4958 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:39:19.152801 4955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:39:19.152777 4956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:39:19.153055 4950 server_base.cc:1034] running on GCE node
I20260806 11:39:19.153858 4950 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:39:19.155493 4950 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:39:19.156785 4950 hybrid_clock.cc:648] HybridClock initialized: now 1786016359156760 us; error 52 us; skew 500 ppm
Aug 06 11:39:19 dist-test-slave-7c35 krb5kdc[3793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016359, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:39:19.160235 4950 init.cc:377] Logged in from keytab as kudu/127.21.14.254@KRBTEST.COM (short username kudu)
I20260806 11:39:19.161579 4950 webserver.cc:460] Webserver started at http://127.21.14.254:38567/ using document root <none> and password file <none>
I20260806 11:39:19.162381 4950 fs_manager.cc:359] Metadata directory not provided
I20260806 11:39:19.162469 4950 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:39:19.167357 4950 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 11:39:19.170049 4965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:39:19.171785 4950 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 11:39:19.172071 4950 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/master-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/master-0/wal
uuid: "beea7743cac049cfb113cc825011d29e"
format_stamp: "Formatted at 2026-08-06 11:39:16 on dist-test-slave-7c35"
server_key: "c8ab8bf50e62fe9626dfaa8e07ca149d"
server_key_iv: "05fa07ec7bb16e4ef5d888862e737b13"
server_key_version: "encryptionkey@0"
I20260806 11:39:19.173142 4950 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:39:19.196938 4950 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:39:19.200482 4950 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:39:19.200762 4950 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:39:19.209750 4950 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.254:37871
I20260806 11:39:19.210088 5020 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.254:37871 every 8 connection(s)
I20260806 11:39:19.211246 4950 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
I20260806 11:39:19.216043 5021 sys_catalog.cc:263] Verifying existing consensus state
I20260806 11:39:19.216774 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 4950
I20260806 11:39:19.217316 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.193:43653
--local_ip_for_outbound_sockets=127.21.14.193
--tserver_master_addrs=127.21.14.254:37871
--webserver_port=35725
--webserver_interface=127.21.14.193
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab
--principal=kudu/127.21.14.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:45535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 11:39:19.218498 5021 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P beea7743cac049cfb113cc825011d29e: Bootstrap starting.
I20260806 11:39:19.238343 5021 log.cc:824] T 00000000000000000000000000000000 P beea7743cac049cfb113cc825011d29e: Log is configured to *not* fsync() on all Append() calls
I20260806 11:39:19.254019 5021 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P beea7743cac049cfb113cc825011d29e: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 11:39:19.254616 5021 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P beea7743cac049cfb113cc825011d29e: Bootstrap complete.
I20260806 11:39:19.260718 5021 raft_consensus.cc:348] T 00000000000000000000000000000000 P beea7743cac049cfb113cc825011d29e [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beea7743cac049cfb113cc825011d29e" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 37871 } }
I20260806 11:39:19.261327 5021 raft_consensus.cc:729] T 00000000000000000000000000000000 P beea7743cac049cfb113cc825011d29e [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: beea7743cac049cfb113cc825011d29e, State: Initialized, Role: FOLLOWER
I20260806 11:39:19.261933 5021 consensus_queue.cc:260] T 00000000000000000000000000000000 P beea7743cac049cfb113cc825011d29e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beea7743cac049cfb113cc825011d29e" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 37871 } }
I20260806 11:39:19.262111 5021 raft_consensus.cc:388] T 00000000000000000000000000000000 P beea7743cac049cfb113cc825011d29e [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 11:39:19.262181 5021 raft_consensus.cc:482] T 00000000000000000000000000000000 P beea7743cac049cfb113cc825011d29e [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 11:39:19.262286 5021 raft_consensus.cc:3037] T 00000000000000000000000000000000 P beea7743cac049cfb113cc825011d29e [term 2 FOLLOWER]: Advancing to term 3
I20260806 11:39:19.265071 5021 raft_consensus.cc:504] T 00000000000000000000000000000000 P beea7743cac049cfb113cc825011d29e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beea7743cac049cfb113cc825011d29e" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 37871 } }
I20260806 11:39:19.265473 5021 leader_election.cc:302] T 00000000000000000000000000000000 P beea7743cac049cfb113cc825011d29e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: beea7743cac049cfb113cc825011d29e; no voters:
I20260806 11:39:19.265879 5021 leader_election.cc:288] T 00000000000000000000000000000000 P beea7743cac049cfb113cc825011d29e [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 11:39:19.266076 5025 raft_consensus.cc:2781] T 00000000000000000000000000000000 P beea7743cac049cfb113cc825011d29e [term 3 FOLLOWER]: Leader election won for term 3
I20260806 11:39:19.266374 5025 raft_consensus.cc:686] T 00000000000000000000000000000000 P beea7743cac049cfb113cc825011d29e [term 3 LEADER]: Becoming Leader. State: Replica: beea7743cac049cfb113cc825011d29e, State: Running, Role: LEADER
I20260806 11:39:19.266753 5025 consensus_queue.cc:237] T 00000000000000000000000000000000 P beea7743cac049cfb113cc825011d29e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beea7743cac049cfb113cc825011d29e" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 37871 } }
I20260806 11:39:19.267354 5021 sys_catalog.cc:564] T 00000000000000000000000000000000 P beea7743cac049cfb113cc825011d29e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 11:39:19.268275 5027 sys_catalog.cc:455] T 00000000000000000000000000000000 P beea7743cac049cfb113cc825011d29e [sys.catalog]: SysCatalogTable state changed. Reason: New leader beea7743cac049cfb113cc825011d29e. Latest consensus state: current_term: 3 leader_uuid: "beea7743cac049cfb113cc825011d29e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beea7743cac049cfb113cc825011d29e" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 37871 } } }
I20260806 11:39:19.268456 5027 sys_catalog.cc:458] T 00000000000000000000000000000000 P beea7743cac049cfb113cc825011d29e [sys.catalog]: This master's current role is: LEADER
I20260806 11:39:19.268800 5026 sys_catalog.cc:455] T 00000000000000000000000000000000 P beea7743cac049cfb113cc825011d29e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "beea7743cac049cfb113cc825011d29e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beea7743cac049cfb113cc825011d29e" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 37871 } } }
I20260806 11:39:19.268885 5026 sys_catalog.cc:458] T 00000000000000000000000000000000 P beea7743cac049cfb113cc825011d29e [sys.catalog]: This master's current role is: LEADER
I20260806 11:39:19.269243 5034 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 11:39:19.274941 5034 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 11:39:19.275708 5034 catalog_manager.cc:1259] Loaded cluster ID: 29f8b5ac6b7845f9a0cc9327663aa7ac
I20260806 11:39:19.275784 5034 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260806 11:39:19.276984 5034 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fbf0f8cb9ad google::LogMessage::Fail() at ??:0
@ 0x7fbf0f8ce88a google::LogMessage::SendToLog() at ??:0
@ 0x7fbf0f8cb4de google::LogMessage::Flush() at ??:0
@ 0x7fbf0f8cca19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fbf1a20afdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7fbf1a20bd2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fbf1a208579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7fbf1a24c640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fbf11b1d3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fbf11b1dc83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7fbf11b1f3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fbf11b0e512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fbf131526db start_thread at ??:0
@ 0x7fbf0e3c961f clone at ??:0
W20260806 11:39:19.380738 5023 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:39:19.381038 5023 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:39:19.381086 5023 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:39:19.385409 5023 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:39:19.385484 5023 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:39:19.385582 5023 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.193
I20260806 11:39:19.391367 5023 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:45535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab
--principal=kudu/127.21.14.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.193:43653
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.14.193
--webserver_port=35725
--tserver_master_addrs=127.21.14.254:37871
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5023
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.193
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:39:19.392602 5023 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:39:19.393689 5023 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:39:19.405607 5067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:39:19.405457 5064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:39:19.405457 5065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:39:19.406229 5023 server_base.cc:1034] running on GCE node
I20260806 11:39:19.406739 5023 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:39:19.407502 5023 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:39:19.408730 5023 hybrid_clock.cc:648] HybridClock initialized: now 1786016359408690 us; error 59 us; skew 500 ppm
Aug 06 11:39:19 dist-test-slave-7c35 krb5kdc[3793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016359, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:39:19.411935 5023 init.cc:377] Logged in from keytab as kudu/127.21.14.193@KRBTEST.COM (short username kudu)
I20260806 11:39:19.413064 5023 webserver.cc:460] Webserver started at http://127.21.14.193:35725/ using document root <none> and password file <none>
I20260806 11:39:19.413712 5023 fs_manager.cc:359] Metadata directory not provided
I20260806 11:39:19.413810 5023 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:39:19.418285 5023 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 11:39:19.420948 5074 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:39:19.422438 5023 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 11:39:19.422598 5023 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-0/wal
uuid: "6ce2bc1867f74eb6b113467463a9430a"
format_stamp: "Formatted at 2026-08-06 11:39:16 on dist-test-slave-7c35"
server_key: "26832165400930665ce02c708c257e99"
server_key_iv: "8493c9f433424ba15d78243c5433e296"
server_key_version: "encryptionkey@0"
I20260806 11:39:19.423287 5023 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:39:19.446067 5023 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:39:19.449733 5023 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:39:19.449970 5023 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:39:19.450728 5023 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:39:19.452047 5081 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 11:39:19.456705 5023 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 11:39:19.456792 5023 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260806 11:39:19.456852 5023 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 11:39:19.458742 5023 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 11:39:19.458817 5023 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260806 11:39:19.459167 5081 tablet_bootstrap.cc:492] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 6ce2bc1867f74eb6b113467463a9430a: Bootstrap starting.
I20260806 11:39:19.470676 5023 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.193:43653
I20260806 11:39:19.471042 5197 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.193:43653 every 8 connection(s)
I20260806 11:39:19.472265 5023 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
I20260806 11:39:19.476817 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 5023
I20260806 11:39:19.477460 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.194:37571
--local_ip_for_outbound_sockets=127.21.14.194
--tserver_master_addrs=127.21.14.254:37871
--webserver_port=42217
--webserver_interface=127.21.14.194
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab
--principal=kudu/127.21.14.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:45535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 11:39:19.482390 5081 log.cc:824] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 6ce2bc1867f74eb6b113467463a9430a: Log is configured to *not* fsync() on all Append() calls
I20260806 11:39:19.490940 5081 tablet_bootstrap.cc:492] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 6ce2bc1867f74eb6b113467463a9430a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 11:39:19.491410 5081 tablet_bootstrap.cc:492] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 6ce2bc1867f74eb6b113467463a9430a: Bootstrap complete.
I20260806 11:39:19.492905 5081 ts_tablet_manager.cc:1397] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 6ce2bc1867f74eb6b113467463a9430a: Time spent bootstrapping tablet: real 0.034s user 0.019s sys 0.012s
I20260806 11:39:19.495967 5081 raft_consensus.cc:348] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 6ce2bc1867f74eb6b113467463a9430a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4fc782bf02c45ea9f7ba7a86862f929" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 45493 } } peers { permanent_uuid: "32e458de791a4f01ac55419385005fa0" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37571 } } peers { permanent_uuid: "6ce2bc1867f74eb6b113467463a9430a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 43653 } }
I20260806 11:39:19.496454 5081 raft_consensus.cc:729] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 6ce2bc1867f74eb6b113467463a9430a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ce2bc1867f74eb6b113467463a9430a, State: Initialized, Role: FOLLOWER
I20260806 11:39:19.497895 5081 consensus_queue.cc:260] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 6ce2bc1867f74eb6b113467463a9430a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4fc782bf02c45ea9f7ba7a86862f929" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 45493 } } peers { permanent_uuid: "32e458de791a4f01ac55419385005fa0" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37571 } } peers { permanent_uuid: "6ce2bc1867f74eb6b113467463a9430a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 43653 } }
I20260806 11:39:19.499176 5081 ts_tablet_manager.cc:1428] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 6ce2bc1867f74eb6b113467463a9430a: Time spent starting tablet: real 0.006s user 0.004s sys 0.003s
W20260806 11:39:19.506376 5200 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:39:19.475122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:37871 (local address 127.21.14.193:38915)
0806 11:39:19.475856 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:19.475926 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0806 11:39:19.477191 (+ 1265us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:39:19.505413 (+ 28222us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:37871: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":227,"threads_started":1}
W20260806 11:39:19.507115 5198 heartbeater.cc:644] Failed to heartbeat to 127.21.14.254:37871 (0 consecutive failures): Network error: Failed to ping master at 127.21.14.254:37871: Client connection negotiation failed: client connection to 127.21.14.254:37871: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 11:39:19.507992 5200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:19.507652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:37871 (local address 127.21.14.193:47253)
0806 11:39:19.507787 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:19.507911 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:37871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260806 11:39:19.508692 5200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:19.508472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:37871 (local address 127.21.14.193:38709)
0806 11:39:19.508579 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:19.508628 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:37871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260806 11:39:19.508965 5198 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 11:39:19.620389 5201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:39:19.620841 5201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:39:19.620949 5201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:39:19.626534 5201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:39:19.626624 5201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:39:19.626713 5201 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.194
I20260806 11:39:19.633181 5201 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:45535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab
--principal=kudu/127.21.14.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.194:37571
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.14.194
--webserver_port=42217
--tserver_master_addrs=127.21.14.254:37871
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5201
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.194
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:39:19.634509 5201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:39:19.635710 5201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:39:19.646585 5214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:39:19.646585 5212 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:39:19.646621 5210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:39:19.646647 5201 server_base.cc:1034] running on GCE node
I20260806 11:39:19.647374 5201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:39:19.648131 5201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:39:19.649358 5201 hybrid_clock.cc:648] HybridClock initialized: now 1786016359649337 us; error 37 us; skew 500 ppm
Aug 06 11:39:19 dist-test-slave-7c35 krb5kdc[3793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016359, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:39:19.652563 5201 init.cc:377] Logged in from keytab as kudu/127.21.14.194@KRBTEST.COM (short username kudu)
I20260806 11:39:19.653534 5201 webserver.cc:460] Webserver started at http://127.21.14.194:42217/ using document root <none> and password file <none>
I20260806 11:39:19.654161 5201 fs_manager.cc:359] Metadata directory not provided
I20260806 11:39:19.654261 5201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:39:19.658639 5201 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 11:39:19.661119 5221 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:39:19.662552 5201 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 11:39:19.662729 5201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-1/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-1/wal
uuid: "32e458de791a4f01ac55419385005fa0"
format_stamp: "Formatted at 2026-08-06 11:39:17 on dist-test-slave-7c35"
server_key: "938466b98f9733c6a445968b4733bdb9"
server_key_iv: "a7e538923c63b93b4c6d1effc5fb4cc6"
server_key_version: "encryptionkey@0"
I20260806 11:39:19.663493 5201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:39:19.674078 5201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:39:19.678216 5201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:39:19.678447 5201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:39:19.679191 5201 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:39:19.680083 5201 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:39:19.680151 5201 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:39:19.680222 5201 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:39:19.680267 5201 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:39:19.689837 5201 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.194:37571
I20260806 11:39:19.689935 5340 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.194:37571 every 8 connection(s)
I20260806 11:39:19.691129 5201 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
I20260806 11:39:19.695958 5344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:19.693642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:37871 (local address 127.21.14.194:53781)
0806 11:39:19.694243 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:19.694383 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:37871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":159,"threads_started":1}
W20260806 11:39:19.696606 5341 heartbeater.cc:644] Failed to heartbeat to 127.21.14.254:37871 (0 consecutive failures): Network error: Failed to ping master at 127.21.14.254:37871: Client connection negotiation failed: client connection to 127.21.14.254:37871: connect: Connection refused (error 111)
I20260806 11:39:19.697348 5344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:19.697091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:37871 (local address 127.21.14.194:35187)
0806 11:39:19.697218 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:19.697284 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:37871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260806 11:39:19.698036 5344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:19.697829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:37871 (local address 127.21.14.194:33787)
0806 11:39:19.697939 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:19.697984 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:37871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260806 11:39:19.698225 5341 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 11:39:19.699357 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 5201
I20260806 11:39:19.699796 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.195:45493
--local_ip_for_outbound_sockets=127.21.14.195
--tserver_master_addrs=127.21.14.254:37871
--webserver_port=39387
--webserver_interface=127.21.14.195
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab
--principal=kudu/127.21.14.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:45535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:39:19.823525 5345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:39:19.823841 5345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:39:19.823889 5345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:39:19.828367 5345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:39:19.828444 5345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:39:19.828545 5345 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.195
I20260806 11:39:19.834327 5345 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:45535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab
--principal=kudu/127.21.14.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.195:45493
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.14.195
--webserver_port=39387
--tserver_master_addrs=127.21.14.254:37871
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5345
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.195
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:39:19.835615 5345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:39:19.836764 5345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 11:39:19.847595 5345 server_base.cc:1034] running on GCE node
W20260806 11:39:19.847648 5351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:39:19.847648 5350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:39:19.847784 5355 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:39:19.848281 5345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:39:19.849004 5345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:39:19.850180 5345 hybrid_clock.cc:648] HybridClock initialized: now 1786016359850142 us; error 55 us; skew 500 ppm
Aug 06 11:39:19 dist-test-slave-7c35 krb5kdc[3793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016359, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:39:19.853175 5345 init.cc:377] Logged in from keytab as kudu/127.21.14.195@KRBTEST.COM (short username kudu)
I20260806 11:39:19.854243 5345 webserver.cc:460] Webserver started at http://127.21.14.195:39387/ using document root <none> and password file <none>
I20260806 11:39:19.854928 5345 fs_manager.cc:359] Metadata directory not provided
I20260806 11:39:19.855026 5345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:39:19.859287 5345 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 11:39:19.861752 5362 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:39:19.863241 5345 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 11:39:19.863396 5345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-2/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-2/wal
uuid: "f4fc782bf02c45ea9f7ba7a86862f929"
format_stamp: "Formatted at 2026-08-06 11:39:17 on dist-test-slave-7c35"
server_key: "559653b6b7a910b2ca05056057139afc"
server_key_iv: "0b8e20d7bf224591b44bde6d71b8ab90"
server_key_version: "encryptionkey@0"
I20260806 11:39:19.864030 5345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:39:19.884189 5345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:39:19.887920 5345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:39:19.888181 5345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:39:19.888849 5345 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:39:19.889763 5345 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:39:19.889835 5345 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:39:19.889907 5345 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:39:19.889948 5345 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:39:19.899820 5345 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.195:45493
I20260806 11:39:19.899837 5482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.195:45493 every 8 connection(s)
I20260806 11:39:19.900921 5345 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
I20260806 11:39:19.905237 5485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:19.903320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:37871 (local address 127.21.14.195:57197)
0806 11:39:19.903871 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:19.904003 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:37871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":159,"threads_started":1}
W20260806 11:39:19.905766 5483 heartbeater.cc:644] Failed to heartbeat to 127.21.14.254:37871 (0 consecutive failures): Network error: Failed to ping master at 127.21.14.254:37871: Client connection negotiation failed: client connection to 127.21.14.254:37871: connect: Connection refused (error 111)
I20260806 11:39:19.906452 5485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:19.906215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:37871 (local address 127.21.14.195:56433)
0806 11:39:19.906333 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:19.906389 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:37871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260806 11:39:19.907305 5485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:19.907027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:37871 (local address 127.21.14.195:36329)
0806 11:39:19.907182 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:19.907239 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:37871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
W20260806 11:39:19.907553 5483 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 11:39:19.908471 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 5345
I20260806 11:39:20.510172 5488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:20.509645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:37871 (local address 127.21.14.193:46713)
0806 11:39:20.509945 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:20.510029 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:37871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":101,"threads_started":1}
I20260806 11:39:20.699481 5489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:20.698989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:37871 (local address 127.21.14.194:44759)
0806 11:39:20.699283 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:20.699351 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:37871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":92,"threads_started":1}
I20260806 11:39:20.908764 5490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:20.908268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:37871 (local address 127.21.14.195:56027)
0806 11:39:20.908578 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:20.908646 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:37871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":94,"threads_started":1}
I20260806 11:39:21.063275 5491 raft_consensus.cc:482] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 6ce2bc1867f74eb6b113467463a9430a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 11:39:21.063511 5491 raft_consensus.cc:504] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 6ce2bc1867f74eb6b113467463a9430a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4fc782bf02c45ea9f7ba7a86862f929" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 45493 } } peers { permanent_uuid: "32e458de791a4f01ac55419385005fa0" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37571 } } peers { permanent_uuid: "6ce2bc1867f74eb6b113467463a9430a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 43653 } }
I20260806 11:39:21.064779 5491 leader_election.cc:288] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 6ce2bc1867f74eb6b113467463a9430a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f4fc782bf02c45ea9f7ba7a86862f929 (127.21.14.195:45493), 32e458de791a4f01ac55419385005fa0 (127.21.14.194:37571)
Aug 06 11:39:21 dist-test-slave-7c35 krb5kdc[3793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016359, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.193@KRBTEST.COM for kudu/127.21.14.195@KRBTEST.COM
Aug 06 11:39:21 dist-test-slave-7c35 krb5kdc[3793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016359, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.193@KRBTEST.COM for kudu/127.21.14.194@KRBTEST.COM
I20260806 11:39:21.090396 5492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:21.065124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.194:37571 (local address 127.21.14.193:48129)
0806 11:39:21.065878 (+ 754us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:21.065890 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 11:39:21.065973 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:39:21.067877 (+ 1904us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:39:21.067976 (+ 99us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:39:21.068609 (+ 633us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:39:21.069552 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:21.069582 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:21.071681 (+ 2099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:21.071685 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:21.073703 (+ 2018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:21.073719 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:21.074962 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:21.074967 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:21.075332 (+ 365us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:21.082296 (+ 6964us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:39:21.082335 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:39:21.085109 (+ 2774us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:39:21.087709 (+ 2600us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:39:21.087712 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:39:21.087713 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:39:21.087985 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:39:21.088402 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:39:21.088405 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:39:21.088407 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:39:21.088570 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:39:21.089291 (+ 721us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:39:21.089298 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:39:21.089567 (+ 269us) client_negotiation.cc:769] Sending connection context
0806 11:39:21.089769 (+ 202us) client_negotiation.cc:240] Negotiation successful
0806 11:39:21.090195 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":679,"mutex_wait_us":478,"spinlock_wait_cycles":33664,"thread_start_us":66,"threads_started":1}
I20260806 11:39:21.090411 5494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:21.065259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:54531 (local address 127.21.14.195:45493)
0806 11:39:21.065611 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:21.065639 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:21.065692 (+ 53us) server_negotiation.cc:407] Connection header received
0806 11:39:21.066742 (+ 1050us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:21.066754 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:21.067291 (+ 537us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:21.067665 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:39:21.069774 (+ 2109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:21.071470 (+ 1696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:21.073938 (+ 2468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:21.074735 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:21.074963 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:21.085218 (+ 10255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:39:21.085228 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:39:21.085251 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:39:21.085300 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:39:21.087357 (+ 2057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:21.088099 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:21.088108 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:21.088117 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:21.088281 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:21.088682 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:21.088686 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:21.088688 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:21.089164 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:39:21.089307 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:21.089890 (+ 583us) server_negotiation.cc:299] Negotiation successful
0806 11:39:21.090186 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":155,"threads_started":1}
I20260806 11:39:21.090413 5495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:21.065416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:48129 (local address 127.21.14.194:37571)
0806 11:39:21.065784 (+ 368us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:21.065807 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:21.065975 (+ 168us) server_negotiation.cc:407] Connection header received
0806 11:39:21.066837 (+ 862us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:21.066872 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:21.067290 (+ 418us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:21.067702 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:39:21.069774 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:21.071470 (+ 1696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:21.073925 (+ 2455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:21.074735 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:21.074970 (+ 235us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:21.085378 (+ 10408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:39:21.085384 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:39:21.085397 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:39:21.085440 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:39:21.087560 (+ 2120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:21.088152 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:21.088158 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:21.088162 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:21.088289 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:21.088704 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:21.088707 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:21.088708 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:21.089158 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:39:21.089290 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:21.089890 (+ 600us) server_negotiation.cc:299] Negotiation successful
0806 11:39:21.090212 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":141,"threads_started":1}
I20260806 11:39:21.090461 5493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:21.065014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.195:45493 (local address 127.21.14.193:54531)
0806 11:39:21.065513 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:21.065535 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 11:39:21.065695 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:39:21.067877 (+ 2182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:39:21.067895 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:39:21.068609 (+ 714us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:39:21.069552 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:21.069582 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:21.071681 (+ 2099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:21.071685 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:21.073703 (+ 2018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:21.073719 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:21.074962 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:21.074967 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:21.075330 (+ 363us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:21.082296 (+ 6966us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:39:21.082340 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:39:21.084833 (+ 2493us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:39:21.087522 (+ 2689us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:39:21.087531 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:39:21.087557 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:39:21.087942 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:39:21.088399 (+ 457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:39:21.088403 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:39:21.088405 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:39:21.088579 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:39:21.089305 (+ 726us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:39:21.089310 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:39:21.089566 (+ 256us) client_negotiation.cc:769] Sending connection context
0806 11:39:21.089769 (+ 203us) client_negotiation.cc:240] Negotiation successful
0806 11:39:21.090279 (+ 510us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":410,"mutex_wait_us":651,"spinlock_wait_cycles":30208,"thread_start_us":107,"threads_started":1}
W20260806 11:39:21.094202 5076 leader_election.cc:341] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 6ce2bc1867f74eb6b113467463a9430a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f4fc782bf02c45ea9f7ba7a86862f929 (127.21.14.195:45493): Not found: Tablet not found: 6a8e14e2573b4ddc98d0fe9fb885fcf8
W20260806 11:39:21.094434 5078 leader_election.cc:341] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 6ce2bc1867f74eb6b113467463a9430a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 32e458de791a4f01ac55419385005fa0 (127.21.14.194:37571): Not found: Tablet not found: 6a8e14e2573b4ddc98d0fe9fb885fcf8
I20260806 11:39:21.094529 5078 leader_election.cc:302] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 6ce2bc1867f74eb6b113467463a9430a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ce2bc1867f74eb6b113467463a9430a; no voters: 32e458de791a4f01ac55419385005fa0, f4fc782bf02c45ea9f7ba7a86862f929
I20260806 11:39:21.094802 5491 raft_consensus.cc:2726] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 6ce2bc1867f74eb6b113467463a9430a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 11:39:21.511519 5493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:21.511194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:37871 (local address 127.21.14.193:38231)
0806 11:39:21.511349 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:21.511423 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:37871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260806 11:39:21.700855 5496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:21.700333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:37871 (local address 127.21.14.194:32945)
0806 11:39:21.700673 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:21.700746 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:37871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":114,"threads_started":1}
I20260806 11:39:21.910089 5497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:21.909606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:37871 (local address 127.21.14.195:60579)
0806 11:39:21.909891 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:21.909960 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:37871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":83,"threads_started":1}
I20260806 11:39:22.512933 5498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:22.512395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:37871 (local address 127.21.14.193:57605)
0806 11:39:22.512713 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:22.512801 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:37871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":82,"threads_started":1}
I20260806 11:39:22.702320 5499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:22.701773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:37871 (local address 127.21.14.194:43995)
0806 11:39:22.702144 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:22.702211 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:37871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":149,"threads_started":1}
I20260806 11:39:22.911369 5500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:22.910961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:37871 (local address 127.21.14.195:55389)
0806 11:39:22.911211 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:22.911266 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:37871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":76,"threads_started":1}
I20260806 11:39:23.029795 5501 raft_consensus.cc:482] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 6ce2bc1867f74eb6b113467463a9430a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 11:39:23.030177 5501 raft_consensus.cc:504] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 6ce2bc1867f74eb6b113467463a9430a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4fc782bf02c45ea9f7ba7a86862f929" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 45493 } } peers { permanent_uuid: "32e458de791a4f01ac55419385005fa0" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37571 } } peers { permanent_uuid: "6ce2bc1867f74eb6b113467463a9430a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 43653 } }
I20260806 11:39:23.030706 5501 leader_election.cc:288] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 6ce2bc1867f74eb6b113467463a9430a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f4fc782bf02c45ea9f7ba7a86862f929 (127.21.14.195:45493), 32e458de791a4f01ac55419385005fa0 (127.21.14.194:37571)
W20260806 11:39:23.031762 5078 leader_election.cc:341] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 6ce2bc1867f74eb6b113467463a9430a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 32e458de791a4f01ac55419385005fa0 (127.21.14.194:37571): Not found: Tablet not found: 6a8e14e2573b4ddc98d0fe9fb885fcf8
W20260806 11:39:23.031879 5076 leader_election.cc:341] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 6ce2bc1867f74eb6b113467463a9430a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f4fc782bf02c45ea9f7ba7a86862f929 (127.21.14.195:45493): Not found: Tablet not found: 6a8e14e2573b4ddc98d0fe9fb885fcf8
I20260806 11:39:23.032001 5076 leader_election.cc:302] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 6ce2bc1867f74eb6b113467463a9430a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ce2bc1867f74eb6b113467463a9430a; no voters: 32e458de791a4f01ac55419385005fa0, f4fc782bf02c45ea9f7ba7a86862f929
I20260806 11:39:23.032203 5501 raft_consensus.cc:2726] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 6ce2bc1867f74eb6b113467463a9430a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 11:39:23.514438 5502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:23.513920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:37871 (local address 127.21.14.193:36163)
0806 11:39:23.514255 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:23.514328 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:37871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":91,"threads_started":1}
I20260806 11:39:23.703708 5503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:23.703238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:37871 (local address 127.21.14.194:50075)
0806 11:39:23.703543 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:23.703601 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:37871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":94,"threads_started":1}
I20260806 11:39:23.912832 5504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:23.912302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:37871 (local address 127.21.14.195:42593)
0806 11:39:23.912650 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:23.912721 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:37871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":84,"threads_started":1}
I20260806 11:39:24.516227 5505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:24.515639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:37871 (local address 127.21.14.193:52953)
0806 11:39:24.516048 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:24.516120 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:37871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":108,"threads_started":1}
I20260806 11:39:24.705168 5506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:24.704672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:37871 (local address 127.21.14.194:54527)
0806 11:39:24.704994 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:24.705061 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:37871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":119,"threads_started":1}
I20260806 11:39:24.914274 5507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:24.913807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:37871 (local address 127.21.14.195:50427)
0806 11:39:24.914089 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:24.914167 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:37871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":84,"threads_started":1}
I20260806 11:39:25.517516 5508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:25.517046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:37871 (local address 127.21.14.193:51011)
0806 11:39:25.517342 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:25.517413 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:37871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":90,"threads_started":1}
I20260806 11:39:25.706678 5509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:25.706165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:37871 (local address 127.21.14.194:59999)
0806 11:39:25.706502 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:25.706570 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:37871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":92,"threads_started":1}
I20260806 11:39:25.915786 5510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:25.915269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:37871 (local address 127.21.14.195:35679)
0806 11:39:25.915608 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:25.915676 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:37871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":110,"threads_started":1}
I20260806 11:39:26.243335 5511 raft_consensus.cc:482] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 6ce2bc1867f74eb6b113467463a9430a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 11:39:26.243546 5511 raft_consensus.cc:504] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 6ce2bc1867f74eb6b113467463a9430a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4fc782bf02c45ea9f7ba7a86862f929" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 45493 } } peers { permanent_uuid: "32e458de791a4f01ac55419385005fa0" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37571 } } peers { permanent_uuid: "6ce2bc1867f74eb6b113467463a9430a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 43653 } }
I20260806 11:39:26.244050 5511 leader_election.cc:288] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 6ce2bc1867f74eb6b113467463a9430a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f4fc782bf02c45ea9f7ba7a86862f929 (127.21.14.195:45493), 32e458de791a4f01ac55419385005fa0 (127.21.14.194:37571)
W20260806 11:39:26.244983 5078 leader_election.cc:341] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 6ce2bc1867f74eb6b113467463a9430a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 32e458de791a4f01ac55419385005fa0 (127.21.14.194:37571): Not found: Tablet not found: 6a8e14e2573b4ddc98d0fe9fb885fcf8
W20260806 11:39:26.245101 5076 leader_election.cc:341] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 6ce2bc1867f74eb6b113467463a9430a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f4fc782bf02c45ea9f7ba7a86862f929 (127.21.14.195:45493): Not found: Tablet not found: 6a8e14e2573b4ddc98d0fe9fb885fcf8
I20260806 11:39:26.245219 5076 leader_election.cc:302] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 6ce2bc1867f74eb6b113467463a9430a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ce2bc1867f74eb6b113467463a9430a; no voters: 32e458de791a4f01ac55419385005fa0, f4fc782bf02c45ea9f7ba7a86862f929
I20260806 11:39:26.245388 5511 raft_consensus.cc:2726] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 6ce2bc1867f74eb6b113467463a9430a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 11:39:26.519075 5512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:26.518497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:37871 (local address 127.21.14.193:52671)
0806 11:39:26.518814 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:26.518961 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:37871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":91,"threads_started":1}
I20260806 11:39:26.708230 5513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:26.707617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:37871 (local address 127.21.14.194:46741)
0806 11:39:26.708054 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:26.708125 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:37871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":84,"threads_started":1}
I20260806 11:39:26.917172 5514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:26.916655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:37871 (local address 127.21.14.195:50741)
0806 11:39:26.917004 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:26.917067 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:37871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":80,"threads_started":1}
I20260806 11:39:27.520499 5515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:27.520005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:37871 (local address 127.21.14.193:49437)
0806 11:39:27.520317 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:27.520386 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:37871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":91,"threads_started":1}
I20260806 11:39:27.709607 5516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:27.709093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:37871 (local address 127.21.14.194:41879)
0806 11:39:27.709405 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:27.709468 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:37871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":109,"threads_started":1}
I20260806 11:39:27.918751 5517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:27.918114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:37871 (local address 127.21.14.195:57111)
0806 11:39:27.918513 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:27.918601 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:37871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":86,"threads_started":1}
I20260806 11:39:28.521796 5518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:28.521340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:37871 (local address 127.21.14.193:60737)
0806 11:39:28.521615 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:28.521677 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:37871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":82,"threads_started":1}
I20260806 11:39:28.710999 5519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:28.710457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:37871 (local address 127.21.14.194:53587)
0806 11:39:28.710746 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:28.710809 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:37871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":84,"threads_started":1}
I20260806 11:39:28.920331 5520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:28.919826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:37871 (local address 127.21.14.195:48613)
0806 11:39:28.920163 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:28.920225 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:37871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":112,"threads_started":1}
I20260806 11:39:29.523267 5521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:29.522784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:37871 (local address 127.21.14.193:57295)
0806 11:39:29.523104 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:29.523165 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:37871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":143,"threads_started":1}
I20260806 11:39:29.712329 5522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:29.711861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:37871 (local address 127.21.14.194:36791)
0806 11:39:29.712161 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:29.712227 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:37871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":96,"threads_started":1}
I20260806 11:39:29.921727 5523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:29.921243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:37871 (local address 127.21.14.195:39991)
0806 11:39:29.921554 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:29.921621 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:37871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":123,"threads_started":1}
I20260806 11:39:30.112512 5524 raft_consensus.cc:482] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 6ce2bc1867f74eb6b113467463a9430a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 11:39:30.112659 5524 raft_consensus.cc:504] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 6ce2bc1867f74eb6b113467463a9430a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4fc782bf02c45ea9f7ba7a86862f929" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 45493 } } peers { permanent_uuid: "32e458de791a4f01ac55419385005fa0" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37571 } } peers { permanent_uuid: "6ce2bc1867f74eb6b113467463a9430a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 43653 } }
I20260806 11:39:30.113099 5524 leader_election.cc:288] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 6ce2bc1867f74eb6b113467463a9430a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f4fc782bf02c45ea9f7ba7a86862f929 (127.21.14.195:45493), 32e458de791a4f01ac55419385005fa0 (127.21.14.194:37571)
W20260806 11:39:30.114022 5076 leader_election.cc:341] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 6ce2bc1867f74eb6b113467463a9430a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f4fc782bf02c45ea9f7ba7a86862f929 (127.21.14.195:45493): Not found: Tablet not found: 6a8e14e2573b4ddc98d0fe9fb885fcf8
W20260806 11:39:30.114142 5078 leader_election.cc:341] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 6ce2bc1867f74eb6b113467463a9430a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 32e458de791a4f01ac55419385005fa0 (127.21.14.194:37571): Not found: Tablet not found: 6a8e14e2573b4ddc98d0fe9fb885fcf8
I20260806 11:39:30.114210 5078 leader_election.cc:302] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 6ce2bc1867f74eb6b113467463a9430a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ce2bc1867f74eb6b113467463a9430a; no voters: 32e458de791a4f01ac55419385005fa0, f4fc782bf02c45ea9f7ba7a86862f929
I20260806 11:39:30.114367 5524 raft_consensus.cc:2726] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 6ce2bc1867f74eb6b113467463a9430a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 11:39:30.524633 5525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:30.524097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:37871 (local address 127.21.14.193:55579)
0806 11:39:30.524412 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:30.524529 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:37871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":82,"threads_started":1}
I20260806 11:39:30.713668 5526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:30.713219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:37871 (local address 127.21.14.194:42551)
0806 11:39:30.713501 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:30.713561 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:37871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":95,"threads_started":1}
I20260806 11:39:30.922961 5527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:30.922477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:37871 (local address 127.21.14.195:52787)
0806 11:39:30.922768 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:30.922825 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:37871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":112,"threads_started":1}
I20260806 11:39:31.525883 5528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:31.525470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:37871 (local address 127.21.14.193:54681)
0806 11:39:31.525722 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:31.525782 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:37871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":79,"threads_started":1}
I20260806 11:39:31.715011 5529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:31.714465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:37871 (local address 127.21.14.194:54359)
0806 11:39:31.714790 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:31.714900 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:37871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":104,"threads_started":1}
I20260806 11:39:31.924506 5530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:31.923988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:37871 (local address 127.21.14.195:53389)
0806 11:39:31.924297 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:31.924394 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:37871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":103,"threads_started":1}
I20260806 11:39:32.527591 5531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:32.526978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:37871 (local address 127.21.14.193:50359)
0806 11:39:32.527356 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:32.527444 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:37871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":100,"threads_started":1}
I20260806 11:39:32.716679 5532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:32.716011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:37871 (local address 127.21.14.194:58809)
0806 11:39:32.716457 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:32.716559 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:37871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":92,"threads_started":1}
I20260806 11:39:32.925947 5533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:32.925477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:37871 (local address 127.21.14.195:48659)
0806 11:39:32.925776 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:32.925843 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:37871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":85,"threads_started":1}
I20260806 11:39:33.529209 5534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:33.528633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:37871 (local address 127.21.14.193:34875)
0806 11:39:33.528969 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:33.529080 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:37871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":97,"threads_started":1}
I20260806 11:39:33.718137 5535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:33.717603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:37871 (local address 127.21.14.194:37353)
0806 11:39:33.717938 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:33.718010 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:37871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":101,"threads_started":1}
I20260806 11:39:33.927505 5536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:33.926998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:37871 (local address 127.21.14.195:54935)
0806 11:39:33.927276 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:33.927339 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:37871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":88,"threads_started":1}
I20260806 11:39:33.981906 5537 raft_consensus.cc:482] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 6ce2bc1867f74eb6b113467463a9430a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 11:39:33.982057 5537 raft_consensus.cc:504] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 6ce2bc1867f74eb6b113467463a9430a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4fc782bf02c45ea9f7ba7a86862f929" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 45493 } } peers { permanent_uuid: "32e458de791a4f01ac55419385005fa0" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37571 } } peers { permanent_uuid: "6ce2bc1867f74eb6b113467463a9430a" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 43653 } }
I20260806 11:39:33.982611 5537 leader_election.cc:288] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 6ce2bc1867f74eb6b113467463a9430a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f4fc782bf02c45ea9f7ba7a86862f929 (127.21.14.195:45493), 32e458de791a4f01ac55419385005fa0 (127.21.14.194:37571)
W20260806 11:39:33.983534 5078 leader_election.cc:341] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 6ce2bc1867f74eb6b113467463a9430a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 32e458de791a4f01ac55419385005fa0 (127.21.14.194:37571): Not found: Tablet not found: 6a8e14e2573b4ddc98d0fe9fb885fcf8
W20260806 11:39:33.983652 5076 leader_election.cc:341] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 6ce2bc1867f74eb6b113467463a9430a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f4fc782bf02c45ea9f7ba7a86862f929 (127.21.14.195:45493): Not found: Tablet not found: 6a8e14e2573b4ddc98d0fe9fb885fcf8
I20260806 11:39:33.983724 5076 leader_election.cc:302] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 6ce2bc1867f74eb6b113467463a9430a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ce2bc1867f74eb6b113467463a9430a; no voters: 32e458de791a4f01ac55419385005fa0, f4fc782bf02c45ea9f7ba7a86862f929
I20260806 11:39:33.983887 5537 raft_consensus.cc:2726] T 6a8e14e2573b4ddc98d0fe9fb885fcf8 P 6ce2bc1867f74eb6b113467463a9430a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 11:39:34.530982 5538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:34.530465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:37871 (local address 127.21.14.193:40989)
0806 11:39:34.530746 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:34.530806 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:37871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":97,"threads_started":1}
I20260806 11:39:34.719779 5539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:34.719274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:37871 (local address 127.21.14.194:52967)
0806 11:39:34.719561 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:34.719653 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:37871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":100,"threads_started":1}
I20260806 11:39:34.912485 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 5023
I20260806 11:39:34.921350 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 5201
I20260806 11:39:34.929078 5540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:34.928596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:37871 (local address 127.21.14.195:38077)
0806 11:39:34.928895 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:34.928961 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:37871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":96,"threads_started":1}
I20260806 11:39:34.930403 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 5345
2026-08-06T11:39:34Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19829 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 11:39:35 dist-test-slave-7c35 krb5kdc[5543](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 11:39:35 dist-test-slave-7c35 krb5kdc[5543](info): set up 2 sockets
Aug 06 11:39:35 dist-test-slave-7c35 krb5kdc[5543](info): commencing operation
krb5kdc: starting...
W20260806 11:39:36.859370 21563 mini_kdc.cc:121] Time spent starting KDC: real 1.904s user 0.001s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Aug 06 11:39:36 dist-test-slave-7c35 krb5kdc[5543](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016376, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T11:39:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T11:39:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.14.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab.
Entry for principal kudu/127.21.14.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab.
I20260806 11:39:37.007472 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:36681
--webserver_interface=127.21.14.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab
--principal=kudu/127.21.14.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:43983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.14.254:36681
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:39:37.126215 5557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:39:37.126605 5557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:39:37.126695 5557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:39:37.131103 5557 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 11:39:37.131214 5557 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 11:39:37.131239 5557 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 11:39:37.131265 5557 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 11:39:37.131311 5557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:39:37.131350 5557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:39:37.131389 5557 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 11:39:37.131423 5557 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 11:39:37.137440 5557 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:43983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.21.14.254:36681
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab
--principal=kudu/127.21.14.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:36681
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.14.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5557
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:39:37.138759 5557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:39:37.139951 5557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:39:37.149772 5565 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:39:37.149796 5563 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:39:37.149796 5562 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:39:37.149843 5557 server_base.cc:1034] running on GCE node
I20260806 11:39:37.150794 5557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:39:37.152210 5557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:39:37.153447 5557 hybrid_clock.cc:648] HybridClock initialized: now 1786016377153442 us; error 55 us; skew 500 ppm
Aug 06 11:39:37 dist-test-slave-7c35 krb5kdc[5543](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016377, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:39:37.156734 5557 init.cc:377] Logged in from keytab as kudu/127.21.14.254@KRBTEST.COM (short username kudu)
I20260806 11:39:37.157763 5557 webserver.cc:460] Webserver started at http://127.21.14.254:36065/ using document root <none> and password file <none>
I20260806 11:39:37.158398 5557 fs_manager.cc:359] Metadata directory not provided
I20260806 11:39:37.158500 5557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:39:37.158752 5557 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:39:37.161026 5557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/master-0/data/instance:
uuid: "62e3c47d9e32427eb798272b95ca13a7"
format_stamp: "Formatted at 2026-08-06 11:39:37 on dist-test-slave-7c35"
server_key: "1e45a4d2d9bff82ef252e9bb15b9e105"
server_key_iv: "04cfce1125d7b46e3c555caf4d012c86"
server_key_version: "encryptionkey@0"
I20260806 11:39:37.161657 5557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/master-0/wal/instance:
uuid: "62e3c47d9e32427eb798272b95ca13a7"
format_stamp: "Formatted at 2026-08-06 11:39:37 on dist-test-slave-7c35"
server_key: "1e45a4d2d9bff82ef252e9bb15b9e105"
server_key_iv: "04cfce1125d7b46e3c555caf4d012c86"
server_key_version: "encryptionkey@0"
I20260806 11:39:37.165907 5557 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260806 11:39:37.168932 5572 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:39:37.170336 5557 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 11:39:37.170518 5557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/master-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/master-0/wal
uuid: "62e3c47d9e32427eb798272b95ca13a7"
format_stamp: "Formatted at 2026-08-06 11:39:37 on dist-test-slave-7c35"
server_key: "1e45a4d2d9bff82ef252e9bb15b9e105"
server_key_iv: "04cfce1125d7b46e3c555caf4d012c86"
server_key_version: "encryptionkey@0"
I20260806 11:39:37.170660 5557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:39:37.181751 5557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:39:37.185032 5557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:39:37.185263 5557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:39:37.193130 5557 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.254:36681
I20260806 11:39:37.193451 5630 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.254:36681 every 8 connection(s)
I20260806 11:39:37.194294 5557 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
I20260806 11:39:37.195950 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 5557
I20260806 11:39:37.196115 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/master-0/wal/instance
I20260806 11:39:37.196571 21563 external_mini_cluster.cc:1428] Setting key 346f8ef8f395d204d878c3913f93cb2f
I20260806 11:39:37.197703 5631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:39:37.205262 5631 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62e3c47d9e32427eb798272b95ca13a7: Bootstrap starting.
Aug 06 11:39:37 dist-test-slave-7c35 krb5kdc[5543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016376, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:39:37.208248 5631 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 62e3c47d9e32427eb798272b95ca13a7: Neither blocks nor log segments found. Creating new log.
I20260806 11:39:37.209379 5631 log.cc:824] T 00000000000000000000000000000000 P 62e3c47d9e32427eb798272b95ca13a7: Log is configured to *not* fsync() on all Append() calls
I20260806 11:39:37.212301 5631 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62e3c47d9e32427eb798272b95ca13a7: No bootstrap required, opened a new log
I20260806 11:39:37.213797 5634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:37.198201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45706 (local address 127.21.14.254:36681)
0806 11:39:37.198711 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:37.198732 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:37.198777 (+ 45us) server_negotiation.cc:407] Connection header received
0806 11:39:37.199730 (+ 953us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:37.199758 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:37.200208 (+ 450us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:37.200624 (+ 416us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:39:37.201421 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:37.202959 (+ 1538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:37.203931 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:37.204666 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:37.204846 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:37.207895 (+ 3049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:39:37.207902 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:39:37.207922 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:39:37.207961 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:39:37.210172 (+ 2211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:37.210763 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:37.210770 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:37.210777 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:37.210957 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:37.211278 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:37.211283 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:37.211287 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:37.211848 (+ 561us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:39:37.212041 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:37.212428 (+ 387us) server_negotiation.cc:299] Negotiation successful
0806 11:39:37.212775 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":154,"threads_started":1}
I20260806 11:39:37.216883 5631 raft_consensus.cc:348] T 00000000000000000000000000000000 P 62e3c47d9e32427eb798272b95ca13a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62e3c47d9e32427eb798272b95ca13a7" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 36681 } }
I20260806 11:39:37.217139 5631 raft_consensus.cc:374] T 00000000000000000000000000000000 P 62e3c47d9e32427eb798272b95ca13a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:39:37.217177 5631 raft_consensus.cc:729] T 00000000000000000000000000000000 P 62e3c47d9e32427eb798272b95ca13a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62e3c47d9e32427eb798272b95ca13a7, State: Initialized, Role: FOLLOWER
I20260806 11:39:37.217675 5631 consensus_queue.cc:260] T 00000000000000000000000000000000 P 62e3c47d9e32427eb798272b95ca13a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62e3c47d9e32427eb798272b95ca13a7" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 36681 } }
I20260806 11:39:37.217824 5631 raft_consensus.cc:388] T 00000000000000000000000000000000 P 62e3c47d9e32427eb798272b95ca13a7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 11:39:37.217890 5631 raft_consensus.cc:482] T 00000000000000000000000000000000 P 62e3c47d9e32427eb798272b95ca13a7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 11:39:37.217996 5631 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 62e3c47d9e32427eb798272b95ca13a7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:39:37.219074 5631 raft_consensus.cc:504] T 00000000000000000000000000000000 P 62e3c47d9e32427eb798272b95ca13a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62e3c47d9e32427eb798272b95ca13a7" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 36681 } }
I20260806 11:39:37.219439 5631 leader_election.cc:302] T 00000000000000000000000000000000 P 62e3c47d9e32427eb798272b95ca13a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 62e3c47d9e32427eb798272b95ca13a7; no voters:
I20260806 11:39:37.219796 5631 leader_election.cc:288] T 00000000000000000000000000000000 P 62e3c47d9e32427eb798272b95ca13a7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 11:39:37.220085 5636 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 62e3c47d9e32427eb798272b95ca13a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:39:37.220342 5636 raft_consensus.cc:686] T 00000000000000000000000000000000 P 62e3c47d9e32427eb798272b95ca13a7 [term 1 LEADER]: Becoming Leader. State: Replica: 62e3c47d9e32427eb798272b95ca13a7, State: Running, Role: LEADER
I20260806 11:39:37.220749 5636 consensus_queue.cc:237] T 00000000000000000000000000000000 P 62e3c47d9e32427eb798272b95ca13a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62e3c47d9e32427eb798272b95ca13a7" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 36681 } }
I20260806 11:39:37.221346 5631 sys_catalog.cc:564] T 00000000000000000000000000000000 P 62e3c47d9e32427eb798272b95ca13a7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 11:39:37.222499 5638 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62e3c47d9e32427eb798272b95ca13a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 62e3c47d9e32427eb798272b95ca13a7. Latest consensus state: current_term: 1 leader_uuid: "62e3c47d9e32427eb798272b95ca13a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62e3c47d9e32427eb798272b95ca13a7" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 36681 } } }
I20260806 11:39:37.222661 5638 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62e3c47d9e32427eb798272b95ca13a7 [sys.catalog]: This master's current role is: LEADER
I20260806 11:39:37.223033 5637 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62e3c47d9e32427eb798272b95ca13a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "62e3c47d9e32427eb798272b95ca13a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62e3c47d9e32427eb798272b95ca13a7" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 36681 } } }
I20260806 11:39:37.223136 5637 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62e3c47d9e32427eb798272b95ca13a7 [sys.catalog]: This master's current role is: LEADER
W20260806 11:39:37.226418 5651 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 62e3c47d9e32427eb798272b95ca13a7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 11:39:37.226495 5651 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 11:39:37.226583 5645 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 11:39:37.227415 5645 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 11:39:37.233341 5645 catalog_manager.cc:1347] Generated new cluster ID: 1f5eded52c3a49deab2b799491110e1f
I20260806 11:39:37.233440 5645 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 11:39:37.247978 5645 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 11:39:37.249020 5645 catalog_manager.cc:1504] Loading token signing keys...
I20260806 11:39:37.262127 5645 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 62e3c47d9e32427eb798272b95ca13a7: Generated new TSK 0
I20260806 11:39:37.263139 5645 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.14.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab.
Entry for principal kudu/127.21.14.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab.
I20260806 11:39:37.312913 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.193:0
--local_ip_for_outbound_sockets=127.21.14.193
--webserver_interface=127.21.14.193
--webserver_port=0
--tserver_master_addrs=127.21.14.254:36681
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab
--principal=kudu/127.21.14.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:43983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:39:37.445711 5657 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:39:37.446039 5657 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:39:37.446089 5657 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:39:37.450378 5657 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:39:37.450461 5657 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:39:37.450531 5657 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.193
I20260806 11:39:37.456252 5657 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:43983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab
--principal=kudu/127.21.14.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.14.193
--webserver_port=0
--tserver_master_addrs=127.21.14.254:36681
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5657
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.193
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:39:37.457484 5657 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:39:37.458587 5657 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:39:37.468856 5663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:39:37.468914 5662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:39:37.468917 5665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:39:37.469089 5657 server_base.cc:1034] running on GCE node
I20260806 11:39:37.469676 5657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:39:37.470347 5657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:39:37.471521 5657 hybrid_clock.cc:648] HybridClock initialized: now 1786016377471477 us; error 57 us; skew 500 ppm
Aug 06 11:39:37 dist-test-slave-7c35 krb5kdc[5543](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016377, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:39:37.474531 5657 init.cc:377] Logged in from keytab as kudu/127.21.14.193@KRBTEST.COM (short username kudu)
I20260806 11:39:37.475643 5657 webserver.cc:460] Webserver started at http://127.21.14.193:38685/ using document root <none> and password file <none>
I20260806 11:39:37.476227 5657 fs_manager.cc:359] Metadata directory not provided
I20260806 11:39:37.476291 5657 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:39:37.476467 5657 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:39:37.478776 5657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-0/data/instance:
uuid: "a44ebc4b518a42d5a327cfda159f9635"
format_stamp: "Formatted at 2026-08-06 11:39:37 on dist-test-slave-7c35"
server_key: "f44a7c8cf2c40157cdef5ab6e9d9e01f"
server_key_iv: "400c053b1e170a40f3b2fecf3722e5e7"
server_key_version: "encryptionkey@0"
I20260806 11:39:37.479457 5657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-0/wal/instance:
uuid: "a44ebc4b518a42d5a327cfda159f9635"
format_stamp: "Formatted at 2026-08-06 11:39:37 on dist-test-slave-7c35"
server_key: "f44a7c8cf2c40157cdef5ab6e9d9e01f"
server_key_iv: "400c053b1e170a40f3b2fecf3722e5e7"
server_key_version: "encryptionkey@0"
I20260806 11:39:37.483595 5657 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 11:39:37.486385 5672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:39:37.487783 5657 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 11:39:37.487924 5657 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-0/wal
uuid: "a44ebc4b518a42d5a327cfda159f9635"
format_stamp: "Formatted at 2026-08-06 11:39:37 on dist-test-slave-7c35"
server_key: "f44a7c8cf2c40157cdef5ab6e9d9e01f"
server_key_iv: "400c053b1e170a40f3b2fecf3722e5e7"
server_key_version: "encryptionkey@0"
I20260806 11:39:37.488058 5657 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:39:37.504392 5657 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:39:37.508002 5657 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:39:37.508236 5657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:39:37.508893 5657 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:39:37.509785 5657 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:39:37.509835 5657 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:39:37.509907 5657 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:39:37.509946 5657 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:39:37.519368 5657 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.193:39037
I20260806 11:39:37.519445 5789 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.193:39037 every 8 connection(s)
I20260806 11:39:37.520439 5657 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
I20260806 11:39:37.523905 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 5657
I20260806 11:39:37.524068 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-0/wal/instance
I20260806 11:39:37.524401 21563 external_mini_cluster.cc:1428] Setting key de6056a6d8ee2b7de7c5709cc3f3ca35
Aug 06 11:39:37 dist-test-slave-7c35 krb5kdc[5543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016377, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.193@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.14.194@KRBTEST.COM; defaulting to no policy
I20260806 11:39:37.537714 5634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:37.523064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:55367 (local address 127.21.14.254:36681)
0806 11:39:37.523238 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:37.523243 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:37.524449 (+ 1206us) server_negotiation.cc:407] Connection header received
0806 11:39:37.526488 (+ 2039us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:37.526492 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:37.526561 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:37.526654 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:39:37.528171 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:37.528778 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:37.530148 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:37.530417 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:37.530558 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:37.533527 (+ 2969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:39:37.533531 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:39:37.533533 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:39:37.533560 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:39:37.535225 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:37.535834 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:37.535838 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:37.535839 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:37.535924 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:37.536341 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:37.536345 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:37.536347 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:37.536586 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:39:37.536712 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:37.537306 (+ 594us) server_negotiation.cc:299] Negotiation successful
0806 11:39:37.537502 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.194@KRBTEST.COM" created.
I20260806 11:39:37.538836 5792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:37.523568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:36681 (local address 127.21.14.193:55367)
0806 11:39:37.524262 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:37.524314 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 11:39:37.526231 (+ 1917us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:39:37.526912 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:39:37.526926 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:39:37.527403 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:39:37.527982 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:37.528000 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:37.528935 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:37.528940 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:37.529973 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:37.529984 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:37.530611 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:37.530616 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:37.530913 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:37.531661 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:39:37.531680 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:39:37.533317 (+ 1637us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:39:37.535363 (+ 2046us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:39:37.535371 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:39:37.535389 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:39:37.535699 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:39:37.536048 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:39:37.536052 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:39:37.536054 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:39:37.536221 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:39:37.536711 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:39:37.536720 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:39:37.537023 (+ 303us) client_negotiation.cc:769] Sending connection context
0806 11:39:37.537240 (+ 217us) client_negotiation.cc:240] Negotiation successful
0806 11:39:37.537641 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":186,"threads_started":1}
I20260806 11:39:37.540406 5790 heartbeater.cc:347] Connected to a master server at 127.21.14.254:36681
I20260806 11:39:37.540722 5790 heartbeater.cc:464] Registering TS with master...
I20260806 11:39:37.541301 5790 heartbeater.cc:511] Master 127.21.14.254:36681 requested a full tablet report, sending...
I20260806 11:39:37.543220 5591 ts_manager.cc:194] Registered new tserver with Master: a44ebc4b518a42d5a327cfda159f9635 (127.21.14.193:39037)
I20260806 11:39:37.545060 5591 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.14.193@KRBTEST.COM'} at 127.21.14.193:55367
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab.
Entry for principal kudu/127.21.14.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab.
I20260806 11:39:37.558555 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.194:0
--local_ip_for_outbound_sockets=127.21.14.194
--webserver_interface=127.21.14.194
--webserver_port=0
--tserver_master_addrs=127.21.14.254:36681
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab
--principal=kudu/127.21.14.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:43983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:39:37.690300 5795 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:39:37.690614 5795 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:39:37.690660 5795 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:39:37.695609 5795 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:39:37.695706 5795 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:39:37.695773 5795 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.194
I20260806 11:39:37.701697 5795 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:43983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab
--principal=kudu/127.21.14.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.14.194
--webserver_port=0
--tserver_master_addrs=127.21.14.254:36681
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5795
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.194
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:39:37.703006 5795 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:39:37.704336 5795 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:39:37.716029 5802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:39:37.716149 5803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:39:37.716341 5795 server_base.cc:1034] running on GCE node
W20260806 11:39:37.716029 5805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:39:37.716992 5795 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:39:37.717676 5795 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:39:37.718968 5795 hybrid_clock.cc:648] HybridClock initialized: now 1786016377718942 us; error 29 us; skew 500 ppm
Aug 06 11:39:37 dist-test-slave-7c35 krb5kdc[5543](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016377, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:39:37.722602 5795 init.cc:377] Logged in from keytab as kudu/127.21.14.194@KRBTEST.COM (short username kudu)
I20260806 11:39:37.724026 5795 webserver.cc:460] Webserver started at http://127.21.14.194:36047/ using document root <none> and password file <none>
I20260806 11:39:37.724762 5795 fs_manager.cc:359] Metadata directory not provided
I20260806 11:39:37.724840 5795 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:39:37.725024 5795 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:39:37.727367 5795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-1/data/instance:
uuid: "558d5fa7ae6d48058c7e74f21bc12211"
format_stamp: "Formatted at 2026-08-06 11:39:37 on dist-test-slave-7c35"
server_key: "dd063a7e8a5d6511a2de98e97fa932ec"
server_key_iv: "deb9c4d2f30c84395fdf40128c335312"
server_key_version: "encryptionkey@0"
I20260806 11:39:37.727941 5795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-1/wal/instance:
uuid: "558d5fa7ae6d48058c7e74f21bc12211"
format_stamp: "Formatted at 2026-08-06 11:39:37 on dist-test-slave-7c35"
server_key: "dd063a7e8a5d6511a2de98e97fa932ec"
server_key_iv: "deb9c4d2f30c84395fdf40128c335312"
server_key_version: "encryptionkey@0"
I20260806 11:39:37.732116 5795 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 11:39:37.734817 5813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:39:37.736311 5795 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 11:39:37.736428 5795 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-1/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-1/wal
uuid: "558d5fa7ae6d48058c7e74f21bc12211"
format_stamp: "Formatted at 2026-08-06 11:39:37 on dist-test-slave-7c35"
server_key: "dd063a7e8a5d6511a2de98e97fa932ec"
server_key_iv: "deb9c4d2f30c84395fdf40128c335312"
server_key_version: "encryptionkey@0"
I20260806 11:39:37.736534 5795 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:39:37.751130 5795 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:39:37.756860 5795 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:39:37.757130 5795 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:39:37.758101 5795 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:39:37.759382 5795 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:39:37.759449 5795 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:39:37.759526 5795 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:39:37.759562 5795 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:39:37.769755 5795 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.194:43875
I20260806 11:39:37.769840 5932 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.194:43875 every 8 connection(s)
I20260806 11:39:37.770910 5795 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
I20260806 11:39:37.778087 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 5795
I20260806 11:39:37.778245 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-1/wal/instance
I20260806 11:39:37.778582 21563 external_mini_cluster.cc:1428] Setting key f72c1054a0774f3b88f4b2c3558318c6
Aug 06 11:39:37 dist-test-slave-7c35 krb5kdc[5543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016377, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.194@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:39:37.786459 5634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:37.772993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:45521 (local address 127.21.14.254:36681)
0806 11:39:37.773185 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:37.773190 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:37.774110 (+ 920us) server_negotiation.cc:407] Connection header received
0806 11:39:37.775712 (+ 1602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:37.775714 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:37.775767 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:37.775845 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:39:37.777068 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:37.777624 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:37.779353 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:37.779590 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:37.779676 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:37.782602 (+ 2926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:39:37.782606 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:39:37.782608 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:39:37.782634 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:39:37.784188 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:37.784772 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:37.784776 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:37.784777 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:37.784868 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:37.785256 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:37.785260 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:37.785261 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:37.785466 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:39:37.785588 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:37.786141 (+ 553us) server_negotiation.cc:299] Negotiation successful
0806 11:39:37.786285 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260806 11:39:37.787488 5935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:37.773447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:36681 (local address 127.21.14.194:45521)
0806 11:39:37.773942 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:37.773987 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 11:39:37.775474 (+ 1487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:39:37.776020 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:39:37.776027 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:39:37.776402 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:39:37.776924 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:37.776947 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:37.777798 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:37.777805 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:37.779222 (+ 1417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:37.779234 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:37.779755 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:37.779758 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:37.779971 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:37.780611 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:39:37.780636 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:39:37.782385 (+ 1749us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:39:37.784326 (+ 1941us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:39:37.784332 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:39:37.784352 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:39:37.784651 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:39:37.784970 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:39:37.784973 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:39:37.784975 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:39:37.785148 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:39:37.785584 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:39:37.785591 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:39:37.785879 (+ 288us) client_negotiation.cc:769] Sending connection context
0806 11:39:37.786103 (+ 224us) client_negotiation.cc:240] Negotiation successful
0806 11:39:37.786476 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":131,"threads_started":1}
I20260806 11:39:37.788817 5933 heartbeater.cc:347] Connected to a master server at 127.21.14.254:36681
I20260806 11:39:37.789110 5933 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.21.14.195@KRBTEST.COM; defaulting to no policy
I20260806 11:39:37.789698 5933 heartbeater.cc:511] Master 127.21.14.254:36681 requested a full tablet report, sending...
I20260806 11:39:37.791028 5591 ts_manager.cc:194] Registered new tserver with Master: 558d5fa7ae6d48058c7e74f21bc12211 (127.21.14.194:43875)
I20260806 11:39:37.791787 5591 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.14.194@KRBTEST.COM'} at 127.21.14.194:45521
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab.
Entry for principal kudu/127.21.14.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab.
I20260806 11:39:37.811262 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.195:0
--local_ip_for_outbound_sockets=127.21.14.195
--webserver_interface=127.21.14.195
--webserver_port=0
--tserver_master_addrs=127.21.14.254:36681
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab
--principal=kudu/127.21.14.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:43983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:39:37.931803 5939 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:39:37.932123 5939 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:39:37.932170 5939 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:39:37.936381 5939 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:39:37.936465 5939 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:39:37.936528 5939 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.195
I20260806 11:39:37.942528 5939 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:43983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab
--principal=kudu/127.21.14.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.14.195
--webserver_port=0
--tserver_master_addrs=127.21.14.254:36681
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5939
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.195
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:39:37.943739 5939 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:39:37.944780 5939 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 11:39:37.955246 5939 server_base.cc:1034] running on GCE node
W20260806 11:39:37.955196 5946 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:39:37.955192 5944 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:39:37.955192 5948 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:39:37.955904 5939 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:39:37.956647 5939 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:39:37.957819 5939 hybrid_clock.cc:648] HybridClock initialized: now 1786016377957801 us; error 31 us; skew 500 ppm
Aug 06 11:39:37 dist-test-slave-7c35 krb5kdc[5543](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016377, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:39:37.960930 5939 init.cc:377] Logged in from keytab as kudu/127.21.14.195@KRBTEST.COM (short username kudu)
I20260806 11:39:37.961990 5939 webserver.cc:460] Webserver started at http://127.21.14.195:32799/ using document root <none> and password file <none>
I20260806 11:39:37.962611 5939 fs_manager.cc:359] Metadata directory not provided
I20260806 11:39:37.962706 5939 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:39:37.963011 5939 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:39:37.965216 5939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-2/data/instance:
uuid: "9678ece9548c429ba9076df50c10b444"
format_stamp: "Formatted at 2026-08-06 11:39:37 on dist-test-slave-7c35"
server_key: "6b73a94d75cc4e1fa264b6b69f5d24fe"
server_key_iv: "494109ca392fea40387785f02eebeb71"
server_key_version: "encryptionkey@0"
I20260806 11:39:37.965842 5939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-2/wal/instance:
uuid: "9678ece9548c429ba9076df50c10b444"
format_stamp: "Formatted at 2026-08-06 11:39:37 on dist-test-slave-7c35"
server_key: "6b73a94d75cc4e1fa264b6b69f5d24fe"
server_key_iv: "494109ca392fea40387785f02eebeb71"
server_key_version: "encryptionkey@0"
I20260806 11:39:37.970026 5939 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260806 11:39:37.972887 5957 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:39:37.974267 5939 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 11:39:37.974416 5939 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-2/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-2/wal
uuid: "9678ece9548c429ba9076df50c10b444"
format_stamp: "Formatted at 2026-08-06 11:39:37 on dist-test-slave-7c35"
server_key: "6b73a94d75cc4e1fa264b6b69f5d24fe"
server_key_iv: "494109ca392fea40387785f02eebeb71"
server_key_version: "encryptionkey@0"
I20260806 11:39:37.974546 5939 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:39:37.985405 5939 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:39:37.989045 5939 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:39:37.989284 5939 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:39:37.989917 5939 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:39:37.990799 5939 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:39:37.990909 5939 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:39:37.991003 5939 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:39:37.991061 5939 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:39:38.000626 5939 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.195:36077
I20260806 11:39:38.000653 6079 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.195:36077 every 8 connection(s)
I20260806 11:39:38.001647 5939 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
I20260806 11:39:38.009826 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 5939
I20260806 11:39:38.010025 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-2/wal/instance
I20260806 11:39:38.010356 21563 external_mini_cluster.cc:1428] Setting key 415983675fe66435884e9c9cb5770ed4
Aug 06 11:39:38 dist-test-slave-7c35 krb5kdc[5543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016377, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.195@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:39:38.016746 5634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:38.003761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:56529 (local address 127.21.14.254:36681)
0806 11:39:38.003917 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:38.003921 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:38.004771 (+ 850us) server_negotiation.cc:407] Connection header received
0806 11:39:38.006239 (+ 1468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:38.006243 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:38.006311 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:38.006426 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:39:38.007789 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:38.008351 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:38.009626 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:38.009861 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:38.009981 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:38.012983 (+ 3002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:39:38.012986 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:39:38.012988 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:39:38.013014 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:39:38.014441 (+ 1427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:38.015057 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:38.015061 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:38.015065 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:38.015146 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:38.015528 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:38.015531 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:38.015533 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:38.015737 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:39:38.015843 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:38.016427 (+ 584us) server_negotiation.cc:299] Negotiation successful
0806 11:39:38.016568 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 11:39:38.017650 6082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:38.004081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:36681 (local address 127.21.14.195:56529)
0806 11:39:38.004594 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:38.004642 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 11:39:38.005986 (+ 1344us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:39:38.006626 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:39:38.006634 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:39:38.007076 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:39:38.007613 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:38.007629 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:38.008510 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:38.008519 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:38.009461 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:38.009472 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:38.010039 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:38.010044 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:38.010244 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:38.011007 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:39:38.011035 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:39:38.012794 (+ 1759us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:39:38.014567 (+ 1773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:39:38.014575 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:39:38.014593 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:39:38.014951 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:39:38.015253 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:39:38.015256 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:39:38.015258 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:39:38.015422 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:39:38.015876 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:39:38.015883 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:39:38.016164 (+ 281us) client_negotiation.cc:769] Sending connection context
0806 11:39:38.016363 (+ 199us) client_negotiation.cc:240] Negotiation successful
0806 11:39:38.016723 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":142,"threads_started":1}
I20260806 11:39:38.019023 6080 heartbeater.cc:347] Connected to a master server at 127.21.14.254:36681
I20260806 11:39:38.019309 6080 heartbeater.cc:464] Registering TS with master...
I20260806 11:39:38.019827 6080 heartbeater.cc:511] Master 127.21.14.254:36681 requested a full tablet report, sending...
I20260806 11:39:38.021095 5591 ts_manager.cc:194] Registered new tserver with Master: 9678ece9548c429ba9076df50c10b444 (127.21.14.195:36077)
I20260806 11:39:38.022126 5591 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.14.195@KRBTEST.COM'} at 127.21.14.195:56529
I20260806 11:39:38.025094 21563 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 11:39:38.034896 5634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:38.027164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45722 (local address 127.21.14.254:36681)
0806 11:39:38.027322 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:38.027328 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:38.027531 (+ 203us) server_negotiation.cc:407] Connection header received
0806 11:39:38.027642 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:38.027644 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:38.027687 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:38.027750 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:39:38.028542 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:38.029065 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:38.029893 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:38.030127 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:38.030207 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:38.031767 (+ 1560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:39:38.031770 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:39:38.031772 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:39:38.031798 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:39:38.033204 (+ 1406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:38.033661 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:38.033663 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:38.033665 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:38.033742 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:38.033980 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:38.033982 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:38.033983 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:38.034230 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:39:38.034337 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:38.034547 (+ 210us) server_negotiation.cc:299] Negotiation successful
0806 11:39:38.034652 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 11:39:38.038023 5591 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45722:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 11:39:38.041548 5591 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 11:39:38.057425 6094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:38.051121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.195:36077 (local address 127.0.0.1:41262)
0806 11:39:38.052019 (+ 898us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:38.052032 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 11:39:38.052133 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:39:38.052915 (+ 782us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:39:38.052918 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:39:38.052938 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:39:38.053025 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:38.053033 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:38.054420 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:38.054424 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:38.055843 (+ 1419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:38.055851 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:38.056731 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:38.056734 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:38.056855 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:38.056894 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 11:39:38.057013 (+ 119us) client_negotiation.cc:240] Negotiation successful
0806 11:39:38.057220 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":835,"mutex_wait_us":247,"thread_start_us":101,"threads_started":1}
I20260806 11:39:38.057534 6098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:38.051834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50932 (local address 127.21.14.194:43875)
0806 11:39:38.052389 (+ 555us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:38.052422 (+ 33us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:38.052447 (+ 25us) server_negotiation.cc:407] Connection header received
0806 11:39:38.052521 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:38.052526 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:38.052720 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:38.052886 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:39:38.053307 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:38.054032 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:38.055829 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:38.056716 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:38.056835 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:38.056978 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:38.057244 (+ 266us) server_negotiation.cc:299] Negotiation successful
0806 11:39:38.057364 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":120,"threads_started":1}
I20260806 11:39:38.057790 6096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:38.051121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41262 (local address 127.21.14.195:36077)
0806 11:39:38.052401 (+ 1280us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:38.052415 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:38.052441 (+ 26us) server_negotiation.cc:407] Connection header received
0806 11:39:38.052514 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:38.052518 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:38.052710 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:38.052928 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:39:38.053145 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:38.054242 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:38.055984 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:38.056591 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:38.056725 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:38.056900 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:38.057512 (+ 612us) server_negotiation.cc:299] Negotiation successful
0806 11:39:38.057657 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1166,"thread_start_us":144,"threads_started":1}
I20260806 11:39:38.057884 6093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:38.050923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38388 (local address 127.21.14.193:39037)
0806 11:39:38.051594 (+ 671us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:38.051603 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:38.051963 (+ 360us) server_negotiation.cc:407] Connection header received
0806 11:39:38.052055 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:38.052062 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:38.052315 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:38.052477 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:39:38.052830 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:38.054116 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:38.055967 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:38.056741 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:38.056836 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:38.056978 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:38.057523 (+ 545us) server_negotiation.cc:299] Negotiation successful
0806 11:39:38.057725 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":558,"thread_start_us":138,"threads_started":1}
I20260806 11:39:38.057425 6097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:38.051648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.194:43875 (local address 127.0.0.1:50932)
0806 11:39:38.052045 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:38.052058 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 11:39:38.052147 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:39:38.053079 (+ 932us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:39:38.053083 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:39:38.053103 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:39:38.053195 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:38.053199 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:38.054234 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:38.054236 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:38.055698 (+ 1462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:38.055709 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:38.056985 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:38.056992 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:38.057089 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:38.057110 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 11:39:38.057204 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 11:39:38.057262 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"mutex_wait_us":225,"spinlock_wait_cycles":768,"thread_start_us":105,"threads_started":1}
I20260806 11:39:38.058315 6092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:38.050736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.193:39037 (local address 127.0.0.1:38388)
0806 11:39:38.051608 (+ 872us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:38.051663 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 11:39:38.051809 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:39:38.052485 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:39:38.052491 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:39:38.052516 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:39:38.052683 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:38.052690 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:38.054220 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:38.054223 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:38.055811 (+ 1588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:38.055824 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:38.057079 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:38.057082 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:38.057186 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:38.057213 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 11:39:38.057286 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 11:39:38.057364 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":682,"mutex_wait_us":99,"thread_start_us":105,"threads_started":1}
I20260806 11:39:38.061044 6010 tablet_service.cc:1467] Processing CreateTablet for tablet c2a6854e2d0b4037829eb7d29fbd98bc (DEFAULT_TABLE table=test-table [id=160158dcd525497a920c5d382e47b642]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:39:38.061184 5861 tablet_service.cc:1467] Processing CreateTablet for tablet c2a6854e2d0b4037829eb7d29fbd98bc (DEFAULT_TABLE table=test-table [id=160158dcd525497a920c5d382e47b642]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:39:38.061183 5722 tablet_service.cc:1467] Processing CreateTablet for tablet c2a6854e2d0b4037829eb7d29fbd98bc (DEFAULT_TABLE table=test-table [id=160158dcd525497a920c5d382e47b642]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:39:38.062201 6010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2a6854e2d0b4037829eb7d29fbd98bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:39:38.062245 5861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2a6854e2d0b4037829eb7d29fbd98bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:39:38.062245 5722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2a6854e2d0b4037829eb7d29fbd98bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:39:38.068961 6099 tablet_bootstrap.cc:492] T c2a6854e2d0b4037829eb7d29fbd98bc P 9678ece9548c429ba9076df50c10b444: Bootstrap starting.
I20260806 11:39:38.071251 6100 tablet_bootstrap.cc:492] T c2a6854e2d0b4037829eb7d29fbd98bc P a44ebc4b518a42d5a327cfda159f9635: Bootstrap starting.
I20260806 11:39:38.071758 6101 tablet_bootstrap.cc:492] T c2a6854e2d0b4037829eb7d29fbd98bc P 558d5fa7ae6d48058c7e74f21bc12211: Bootstrap starting.
I20260806 11:39:38.071780 6099 tablet_bootstrap.cc:654] T c2a6854e2d0b4037829eb7d29fbd98bc P 9678ece9548c429ba9076df50c10b444: Neither blocks nor log segments found. Creating new log.
I20260806 11:39:38.073787 6100 tablet_bootstrap.cc:654] T c2a6854e2d0b4037829eb7d29fbd98bc P a44ebc4b518a42d5a327cfda159f9635: Neither blocks nor log segments found. Creating new log.
I20260806 11:39:38.073794 6099 log.cc:824] T c2a6854e2d0b4037829eb7d29fbd98bc P 9678ece9548c429ba9076df50c10b444: Log is configured to *not* fsync() on all Append() calls
I20260806 11:39:38.074481 6101 tablet_bootstrap.cc:654] T c2a6854e2d0b4037829eb7d29fbd98bc P 558d5fa7ae6d48058c7e74f21bc12211: Neither blocks nor log segments found. Creating new log.
I20260806 11:39:38.074759 6100 log.cc:824] T c2a6854e2d0b4037829eb7d29fbd98bc P a44ebc4b518a42d5a327cfda159f9635: Log is configured to *not* fsync() on all Append() calls
I20260806 11:39:38.075472 6101 log.cc:824] T c2a6854e2d0b4037829eb7d29fbd98bc P 558d5fa7ae6d48058c7e74f21bc12211: Log is configured to *not* fsync() on all Append() calls
I20260806 11:39:38.076407 6099 tablet_bootstrap.cc:492] T c2a6854e2d0b4037829eb7d29fbd98bc P 9678ece9548c429ba9076df50c10b444: No bootstrap required, opened a new log
I20260806 11:39:38.076642 6099 ts_tablet_manager.cc:1397] T c2a6854e2d0b4037829eb7d29fbd98bc P 9678ece9548c429ba9076df50c10b444: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260806 11:39:38.076870 6100 tablet_bootstrap.cc:492] T c2a6854e2d0b4037829eb7d29fbd98bc P a44ebc4b518a42d5a327cfda159f9635: No bootstrap required, opened a new log
I20260806 11:39:38.077054 6100 ts_tablet_manager.cc:1397] T c2a6854e2d0b4037829eb7d29fbd98bc P a44ebc4b518a42d5a327cfda159f9635: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260806 11:39:38.077440 6101 tablet_bootstrap.cc:492] T c2a6854e2d0b4037829eb7d29fbd98bc P 558d5fa7ae6d48058c7e74f21bc12211: No bootstrap required, opened a new log
I20260806 11:39:38.077647 6101 ts_tablet_manager.cc:1397] T c2a6854e2d0b4037829eb7d29fbd98bc P 558d5fa7ae6d48058c7e74f21bc12211: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 11:39:38.081315 6101 raft_consensus.cc:348] T c2a6854e2d0b4037829eb7d29fbd98bc P 558d5fa7ae6d48058c7e74f21bc12211 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a44ebc4b518a42d5a327cfda159f9635" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 39037 } } peers { permanent_uuid: "9678ece9548c429ba9076df50c10b444" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 36077 } } peers { permanent_uuid: "558d5fa7ae6d48058c7e74f21bc12211" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 43875 } }
I20260806 11:39:38.081333 6099 raft_consensus.cc:348] T c2a6854e2d0b4037829eb7d29fbd98bc P 9678ece9548c429ba9076df50c10b444 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a44ebc4b518a42d5a327cfda159f9635" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 39037 } } peers { permanent_uuid: "9678ece9548c429ba9076df50c10b444" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 36077 } } peers { permanent_uuid: "558d5fa7ae6d48058c7e74f21bc12211" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 43875 } }
I20260806 11:39:38.081605 6101 raft_consensus.cc:374] T c2a6854e2d0b4037829eb7d29fbd98bc P 558d5fa7ae6d48058c7e74f21bc12211 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:39:38.081624 6099 raft_consensus.cc:374] T c2a6854e2d0b4037829eb7d29fbd98bc P 9678ece9548c429ba9076df50c10b444 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:39:38.081678 6099 raft_consensus.cc:729] T c2a6854e2d0b4037829eb7d29fbd98bc P 9678ece9548c429ba9076df50c10b444 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9678ece9548c429ba9076df50c10b444, State: Initialized, Role: FOLLOWER
I20260806 11:39:38.081691 6101 raft_consensus.cc:729] T c2a6854e2d0b4037829eb7d29fbd98bc P 558d5fa7ae6d48058c7e74f21bc12211 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558d5fa7ae6d48058c7e74f21bc12211, State: Initialized, Role: FOLLOWER
I20260806 11:39:38.081672 6100 raft_consensus.cc:348] T c2a6854e2d0b4037829eb7d29fbd98bc P a44ebc4b518a42d5a327cfda159f9635 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a44ebc4b518a42d5a327cfda159f9635" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 39037 } } peers { permanent_uuid: "9678ece9548c429ba9076df50c10b444" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 36077 } } peers { permanent_uuid: "558d5fa7ae6d48058c7e74f21bc12211" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 43875 } }
I20260806 11:39:38.081923 6100 raft_consensus.cc:374] T c2a6854e2d0b4037829eb7d29fbd98bc P a44ebc4b518a42d5a327cfda159f9635 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:39:38.081992 6100 raft_consensus.cc:729] T c2a6854e2d0b4037829eb7d29fbd98bc P a44ebc4b518a42d5a327cfda159f9635 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a44ebc4b518a42d5a327cfda159f9635, State: Initialized, Role: FOLLOWER
I20260806 11:39:38.082242 6099 consensus_queue.cc:260] T c2a6854e2d0b4037829eb7d29fbd98bc P 9678ece9548c429ba9076df50c10b444 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a44ebc4b518a42d5a327cfda159f9635" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 39037 } } peers { permanent_uuid: "9678ece9548c429ba9076df50c10b444" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 36077 } } peers { permanent_uuid: "558d5fa7ae6d48058c7e74f21bc12211" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 43875 } }
I20260806 11:39:38.082248 6101 consensus_queue.cc:260] T c2a6854e2d0b4037829eb7d29fbd98bc P 558d5fa7ae6d48058c7e74f21bc12211 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a44ebc4b518a42d5a327cfda159f9635" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 39037 } } peers { permanent_uuid: "9678ece9548c429ba9076df50c10b444" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 36077 } } peers { permanent_uuid: "558d5fa7ae6d48058c7e74f21bc12211" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 43875 } }
I20260806 11:39:38.082351 6100 consensus_queue.cc:260] T c2a6854e2d0b4037829eb7d29fbd98bc P a44ebc4b518a42d5a327cfda159f9635 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a44ebc4b518a42d5a327cfda159f9635" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 39037 } } peers { permanent_uuid: "9678ece9548c429ba9076df50c10b444" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 36077 } } peers { permanent_uuid: "558d5fa7ae6d48058c7e74f21bc12211" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 43875 } }
I20260806 11:39:38.083259 5933 heartbeater.cc:503] Master 127.21.14.254:36681 was elected leader, sending a full tablet report...
I20260806 11:39:38.083396 6101 ts_tablet_manager.cc:1428] T c2a6854e2d0b4037829eb7d29fbd98bc P 558d5fa7ae6d48058c7e74f21bc12211: Time spent starting tablet: real 0.006s user 0.001s sys 0.005s
I20260806 11:39:38.083403 6099 ts_tablet_manager.cc:1428] T c2a6854e2d0b4037829eb7d29fbd98bc P 9678ece9548c429ba9076df50c10b444: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260806 11:39:38.083403 6100 ts_tablet_manager.cc:1428] T c2a6854e2d0b4037829eb7d29fbd98bc P a44ebc4b518a42d5a327cfda159f9635: Time spent starting tablet: real 0.006s user 0.000s sys 0.007s
I20260806 11:39:38.083737 5790 heartbeater.cc:503] Master 127.21.14.254:36681 was elected leader, sending a full tablet report...
I20260806 11:39:38.083923 6080 heartbeater.cc:503] Master 127.21.14.254:36681 was elected leader, sending a full tablet report...
W20260806 11:39:38.252748 6081 tablet.cc:2369] T c2a6854e2d0b4037829eb7d29fbd98bc P 9678ece9548c429ba9076df50c10b444: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 11:39:38.272218 5791 tablet.cc:2369] T c2a6854e2d0b4037829eb7d29fbd98bc P a44ebc4b518a42d5a327cfda159f9635: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 11:39:38.272207 5934 tablet.cc:2369] T c2a6854e2d0b4037829eb7d29fbd98bc P 558d5fa7ae6d48058c7e74f21bc12211: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 11:39:38.471352 6105 raft_consensus.cc:482] T c2a6854e2d0b4037829eb7d29fbd98bc P 9678ece9548c429ba9076df50c10b444 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 11:39:38.471609 6105 raft_consensus.cc:504] T c2a6854e2d0b4037829eb7d29fbd98bc P 9678ece9548c429ba9076df50c10b444 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a44ebc4b518a42d5a327cfda159f9635" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 39037 } } peers { permanent_uuid: "9678ece9548c429ba9076df50c10b444" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 36077 } } peers { permanent_uuid: "558d5fa7ae6d48058c7e74f21bc12211" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 43875 } }
I20260806 11:39:38.472929 6105 leader_election.cc:288] T c2a6854e2d0b4037829eb7d29fbd98bc P 9678ece9548c429ba9076df50c10b444 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a44ebc4b518a42d5a327cfda159f9635 (127.21.14.193:39037), 558d5fa7ae6d48058c7e74f21bc12211 (127.21.14.194:43875)
I20260806 11:39:38.476910 6082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:38.473163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.193:39037 (local address 127.21.14.195:59597)
0806 11:39:38.473342 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:38.473359 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 11:39:38.473466 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:39:38.473802 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:39:38.473805 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:39:38.473856 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:39:38.473980 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:38.473986 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:38.474834 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:38.474838 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:38.475746 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:38.475756 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:38.476447 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:38.476451 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:38.476559 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:38.476588 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 11:39:38.476678 (+ 90us) client_negotiation.cc:240] Negotiation successful
0806 11:39:38.476766 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71,"mutex_wait_us":1,"spinlock_wait_cycles":14080}
I20260806 11:39:38.476909 6093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:38.473415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:59597 (local address 127.21.14.193:39037)
0806 11:39:38.473573 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:38.473578 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:38.473600 (+ 22us) server_negotiation.cc:407] Connection header received
0806 11:39:38.473651 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:38.473653 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:38.473693 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:38.473788 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:39:38.474107 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:38.474714 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:38.475871 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:38.476316 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:38.476451 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:38.476522 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:38.476697 (+ 175us) server_negotiation.cc:299] Negotiation successful
0806 11:39:38.476776 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 11:39:38.477128 6108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:38.473286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.194:43875 (local address 127.21.14.195:46567)
0806 11:39:38.473684 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:38.473695 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 11:39:38.473863 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:39:38.474157 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:39:38.474160 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:39:38.474176 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:39:38.474309 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:38.474315 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:38.475169 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:38.475173 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:38.475962 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:38.475977 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:38.476628 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:38.476632 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:38.476738 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:38.476766 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 11:39:38.476900 (+ 134us) client_negotiation.cc:240] Negotiation successful
0806 11:39:38.476984 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"spinlock_wait_cycles":37888,"thread_start_us":99,"threads_started":1}
I20260806 11:39:38.477113 6098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:38.473386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:46567 (local address 127.21.14.194:43875)
0806 11:39:38.473562 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:38.473567 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:38.473800 (+ 233us) server_negotiation.cc:407] Connection header received
0806 11:39:38.473996 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:38.473999 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:38.474046 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:38.474144 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:39:38.474427 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:38.475064 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:38.476093 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:38.476497 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:38.476628 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:38.476684 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:38.476917 (+ 233us) server_negotiation.cc:299] Negotiation successful
0806 11:39:38.476989 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 11:39:38.477802 5743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2a6854e2d0b4037829eb7d29fbd98bc" candidate_uuid: "9678ece9548c429ba9076df50c10b444" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a44ebc4b518a42d5a327cfda159f9635" is_pre_election: true
I20260806 11:39:38.477998 5884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2a6854e2d0b4037829eb7d29fbd98bc" candidate_uuid: "9678ece9548c429ba9076df50c10b444" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558d5fa7ae6d48058c7e74f21bc12211" is_pre_election: true
I20260806 11:39:38.478183 5743 raft_consensus.cc:2454] T c2a6854e2d0b4037829eb7d29fbd98bc P a44ebc4b518a42d5a327cfda159f9635 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9678ece9548c429ba9076df50c10b444 in term 0.
I20260806 11:39:38.478322 5884 raft_consensus.cc:2454] T c2a6854e2d0b4037829eb7d29fbd98bc P 558d5fa7ae6d48058c7e74f21bc12211 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9678ece9548c429ba9076df50c10b444 in term 0.
I20260806 11:39:38.478830 5959 leader_election.cc:302] T c2a6854e2d0b4037829eb7d29fbd98bc P 9678ece9548c429ba9076df50c10b444 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9678ece9548c429ba9076df50c10b444, a44ebc4b518a42d5a327cfda159f9635; no voters:
I20260806 11:39:38.479200 6105 raft_consensus.cc:2781] T c2a6854e2d0b4037829eb7d29fbd98bc P 9678ece9548c429ba9076df50c10b444 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 11:39:38.479291 6105 raft_consensus.cc:482] T c2a6854e2d0b4037829eb7d29fbd98bc P 9678ece9548c429ba9076df50c10b444 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 11:39:38.479336 6105 raft_consensus.cc:3037] T c2a6854e2d0b4037829eb7d29fbd98bc P 9678ece9548c429ba9076df50c10b444 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:39:38.480432 6105 raft_consensus.cc:504] T c2a6854e2d0b4037829eb7d29fbd98bc P 9678ece9548c429ba9076df50c10b444 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a44ebc4b518a42d5a327cfda159f9635" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 39037 } } peers { permanent_uuid: "9678ece9548c429ba9076df50c10b444" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 36077 } } peers { permanent_uuid: "558d5fa7ae6d48058c7e74f21bc12211" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 43875 } }
I20260806 11:39:38.480819 6105 leader_election.cc:288] T c2a6854e2d0b4037829eb7d29fbd98bc P 9678ece9548c429ba9076df50c10b444 [CANDIDATE]: Term 1 election: Requested vote from peers a44ebc4b518a42d5a327cfda159f9635 (127.21.14.193:39037), 558d5fa7ae6d48058c7e74f21bc12211 (127.21.14.194:43875)
I20260806 11:39:38.481221 5743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2a6854e2d0b4037829eb7d29fbd98bc" candidate_uuid: "9678ece9548c429ba9076df50c10b444" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a44ebc4b518a42d5a327cfda159f9635"
I20260806 11:39:38.481344 5884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2a6854e2d0b4037829eb7d29fbd98bc" candidate_uuid: "9678ece9548c429ba9076df50c10b444" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558d5fa7ae6d48058c7e74f21bc12211"
I20260806 11:39:38.481374 5743 raft_consensus.cc:3037] T c2a6854e2d0b4037829eb7d29fbd98bc P a44ebc4b518a42d5a327cfda159f9635 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:39:38.481503 5884 raft_consensus.cc:3037] T c2a6854e2d0b4037829eb7d29fbd98bc P 558d5fa7ae6d48058c7e74f21bc12211 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:39:38.482596 5884 raft_consensus.cc:2454] T c2a6854e2d0b4037829eb7d29fbd98bc P 558d5fa7ae6d48058c7e74f21bc12211 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9678ece9548c429ba9076df50c10b444 in term 1.
I20260806 11:39:38.482599 5743 raft_consensus.cc:2454] T c2a6854e2d0b4037829eb7d29fbd98bc P a44ebc4b518a42d5a327cfda159f9635 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9678ece9548c429ba9076df50c10b444 in term 1.
I20260806 11:39:38.483008 5958 leader_election.cc:302] T c2a6854e2d0b4037829eb7d29fbd98bc P 9678ece9548c429ba9076df50c10b444 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 558d5fa7ae6d48058c7e74f21bc12211, 9678ece9548c429ba9076df50c10b444; no voters:
I20260806 11:39:38.483247 6105 raft_consensus.cc:2781] T c2a6854e2d0b4037829eb7d29fbd98bc P 9678ece9548c429ba9076df50c10b444 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:39:38.483510 6105 raft_consensus.cc:686] T c2a6854e2d0b4037829eb7d29fbd98bc P 9678ece9548c429ba9076df50c10b444 [term 1 LEADER]: Becoming Leader. State: Replica: 9678ece9548c429ba9076df50c10b444, State: Running, Role: LEADER
I20260806 11:39:38.483922 6105 consensus_queue.cc:237] T c2a6854e2d0b4037829eb7d29fbd98bc P 9678ece9548c429ba9076df50c10b444 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a44ebc4b518a42d5a327cfda159f9635" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 39037 } } peers { permanent_uuid: "9678ece9548c429ba9076df50c10b444" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 36077 } } peers { permanent_uuid: "558d5fa7ae6d48058c7e74f21bc12211" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 43875 } }
I20260806 11:39:38.487275 5589 catalog_manager.cc:5495] T c2a6854e2d0b4037829eb7d29fbd98bc P 9678ece9548c429ba9076df50c10b444 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9678ece9548c429ba9076df50c10b444 (127.21.14.195). New cstate: current_term: 1 leader_uuid: "9678ece9548c429ba9076df50c10b444" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a44ebc4b518a42d5a327cfda159f9635" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 39037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9678ece9548c429ba9076df50c10b444" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 36077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "558d5fa7ae6d48058c7e74f21bc12211" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 43875 } health_report { overall_health: UNKNOWN } } }
I20260806 11:39:38.512365 6096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:38.508429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60450 (local address 127.21.14.195:36077)
0806 11:39:38.508595 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:38.508599 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:38.508616 (+ 17us) server_negotiation.cc:407] Connection header received
0806 11:39:38.508746 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:38.508748 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:38.508806 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:38.508948 (+ 142us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 11:39:38.509220 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:38.509750 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:38.510689 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:38.510989 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:38.511106 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:38.511459 (+ 353us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 11:39:38.511937 (+ 478us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 11:39:38.512079 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:38.512143 (+ 64us) server_negotiation.cc:299] Negotiation successful
0806 11:39:38.512207 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 11:39:38.518918 5743 raft_consensus.cc:1261] T c2a6854e2d0b4037829eb7d29fbd98bc P a44ebc4b518a42d5a327cfda159f9635 [term 1 FOLLOWER]: Refusing update from remote peer 9678ece9548c429ba9076df50c10b444: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 11:39:38.518918 5884 raft_consensus.cc:1261] T c2a6854e2d0b4037829eb7d29fbd98bc P 558d5fa7ae6d48058c7e74f21bc12211 [term 1 FOLLOWER]: Refusing update from remote peer 9678ece9548c429ba9076df50c10b444: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 11:39:38.519785 6105 consensus_queue.cc:1035] T c2a6854e2d0b4037829eb7d29fbd98bc P 9678ece9548c429ba9076df50c10b444 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a44ebc4b518a42d5a327cfda159f9635" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 39037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 11:39:38.520037 6109 consensus_queue.cc:1035] T c2a6854e2d0b4037829eb7d29fbd98bc P 9678ece9548c429ba9076df50c10b444 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558d5fa7ae6d48058c7e74f21bc12211" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 43875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 11:39:38.531044 6113 mvcc.cc:205] Tried to move back new op lower bound from 7315523086404853760 to 7315523086272983040. Current Snapshot: MvccSnapshot[applied={T|T < 7315523086404853760}]
I20260806 11:39:38.540068 5589 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45722:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 11:39:38.540294 5589 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45722:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 11:39:38.543177 5589 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 62e3c47d9e32427eb798272b95ca13a7: Sending DeleteTablet for 3 replicas of tablet c2a6854e2d0b4037829eb7d29fbd98bc
I20260806 11:39:38.544128 6010 tablet_service.cc:1514] Processing DeleteTablet for tablet c2a6854e2d0b4037829eb7d29fbd98bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:39:38 UTC) from {username='kudu', principal='kudu/127.21.14.254@KRBTEST.COM'} at 127.0.0.1:41262
I20260806 11:39:38.544109 5722 tablet_service.cc:1514] Processing DeleteTablet for tablet c2a6854e2d0b4037829eb7d29fbd98bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:39:38 UTC) from {username='kudu', principal='kudu/127.21.14.254@KRBTEST.COM'} at 127.0.0.1:38388
I20260806 11:39:38.544215 5861 tablet_service.cc:1514] Processing DeleteTablet for tablet c2a6854e2d0b4037829eb7d29fbd98bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:39:38 UTC) from {username='kudu', principal='kudu/127.21.14.254@KRBTEST.COM'} at 127.0.0.1:50932
I20260806 11:39:38.544870 6126 tablet_replica.cc:330] T c2a6854e2d0b4037829eb7d29fbd98bc P a44ebc4b518a42d5a327cfda159f9635: stopping tablet replica
I20260806 11:39:38.545018 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 5657
I20260806 11:39:38.545323 6126 raft_consensus.cc:2229] T c2a6854e2d0b4037829eb7d29fbd98bc P a44ebc4b518a42d5a327cfda159f9635 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 11:39:38.546183 6124 tablet_replica.cc:330] T c2a6854e2d0b4037829eb7d29fbd98bc P 9678ece9548c429ba9076df50c10b444: stopping tablet replica
I20260806 11:39:38.546500 6124 raft_consensus.cc:2229] T c2a6854e2d0b4037829eb7d29fbd98bc P 9678ece9548c429ba9076df50c10b444 [term 1 LEADER]: Raft consensus shutting down.
I20260806 11:39:38.547506 6125 tablet_replica.cc:330] T c2a6854e2d0b4037829eb7d29fbd98bc P 558d5fa7ae6d48058c7e74f21bc12211: stopping tablet replica
I20260806 11:39:38.547816 6125 raft_consensus.cc:2229] T c2a6854e2d0b4037829eb7d29fbd98bc P 558d5fa7ae6d48058c7e74f21bc12211 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 11:39:38.548174 6125 raft_consensus.cc:2258] T c2a6854e2d0b4037829eb7d29fbd98bc P 558d5fa7ae6d48058c7e74f21bc12211 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 11:39:38.548635 6124 raft_consensus.cc:2258] T c2a6854e2d0b4037829eb7d29fbd98bc P 9678ece9548c429ba9076df50c10b444 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 11:39:38.550134 6125 ts_tablet_manager.cc:1905] T c2a6854e2d0b4037829eb7d29fbd98bc P 558d5fa7ae6d48058c7e74f21bc12211: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 11:39:38.550309 6124 ts_tablet_manager.cc:1905] T c2a6854e2d0b4037829eb7d29fbd98bc P 9678ece9548c429ba9076df50c10b444: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 11:39:38.553591 6124 ts_tablet_manager.cc:1918] T c2a6854e2d0b4037829eb7d29fbd98bc P 9678ece9548c429ba9076df50c10b444: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 11:39:38.553730 6124 log.cc:1196] T c2a6854e2d0b4037829eb7d29fbd98bc P 9678ece9548c429ba9076df50c10b444: Deleting WAL directory at /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-2/wal/wals/c2a6854e2d0b4037829eb7d29fbd98bc
I20260806 11:39:38.553640 6125 ts_tablet_manager.cc:1918] T c2a6854e2d0b4037829eb7d29fbd98bc P 558d5fa7ae6d48058c7e74f21bc12211: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 11:39:38.553968 6125 log.cc:1196] T c2a6854e2d0b4037829eb7d29fbd98bc P 558d5fa7ae6d48058c7e74f21bc12211: Deleting WAL directory at /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-1/wal/wals/c2a6854e2d0b4037829eb7d29fbd98bc
I20260806 11:39:38.554086 6124 ts_tablet_manager.cc:1939] T c2a6854e2d0b4037829eb7d29fbd98bc P 9678ece9548c429ba9076df50c10b444: Deleting consensus metadata
I20260806 11:39:38.554421 6125 ts_tablet_manager.cc:1939] T c2a6854e2d0b4037829eb7d29fbd98bc P 558d5fa7ae6d48058c7e74f21bc12211: Deleting consensus metadata
I20260806 11:39:38.555636 5576 catalog_manager.cc:4841] TS 9678ece9548c429ba9076df50c10b444 (127.21.14.195:36077): tablet c2a6854e2d0b4037829eb7d29fbd98bc (table test-table [id=160158dcd525497a920c5d382e47b642]) successfully deleted
I20260806 11:39:38.555677 5573 catalog_manager.cc:4841] TS 558d5fa7ae6d48058c7e74f21bc12211 (127.21.14.194:43875): tablet c2a6854e2d0b4037829eb7d29fbd98bc (table test-table [id=160158dcd525497a920c5d382e47b642]) successfully deleted
W20260806 11:39:38.561491 5574 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.14.193:39037 (error 108)
I20260806 11:39:38.561673 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 5795
I20260806 11:39:38.563874 6128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:38.563203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.193:39037 (local address 127.0.0.1:53136)
0806 11:39:38.563724 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:38.563783 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.193:39037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":159,"threads_started":1}
W20260806 11:39:38.564047 5574 catalog_manager.cc:4584] TS a44ebc4b518a42d5a327cfda159f9635 (127.21.14.193:39037): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c2a6854e2d0b4037829eb7d29fbd98bc: Network error: Client connection negotiation failed: client connection to 127.21.14.193:39037: connect: Connection refused (error 111)
I20260806 11:39:38.570403 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 5939
I20260806 11:39:38.578392 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 5557
I20260806 11:39:58.588421 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:36681
--webserver_interface=127.21.14.254
--webserver_port=36065
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab
--principal=kudu/127.21.14.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:43983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.14.254:36681
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:39:58.709822 6129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:39:58.710215 6129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:39:58.710295 6129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:39:58.714632 6129 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 11:39:58.714756 6129 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 11:39:58.714782 6129 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 11:39:58.714807 6129 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 11:39:58.714847 6129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:39:58.714910 6129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:39:58.714962 6129 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 11:39:58.715010 6129 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 11:39:58.720935 6129 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:43983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.21.14.254:36681
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab
--principal=kudu/127.21.14.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:36681
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.14.254
--webserver_port=36065
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6129
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:39:58.722229 6129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:39:58.723400 6129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:39:58.738664 6136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:39:58.738664 6138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:39:58.738899 6129 server_base.cc:1034] running on GCE node
W20260806 11:39:58.739111 6135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:39:58.739866 6129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:39:58.741227 6129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:39:58.742427 6129 hybrid_clock.cc:648] HybridClock initialized: now 1786016398742409 us; error 35 us; skew 500 ppm
Aug 06 11:39:58 dist-test-slave-7c35 krb5kdc[5543](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016398, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:39:58.745527 6129 init.cc:377] Logged in from keytab as kudu/127.21.14.254@KRBTEST.COM (short username kudu)
I20260806 11:39:58.746614 6129 webserver.cc:460] Webserver started at http://127.21.14.254:36065/ using document root <none> and password file <none>
I20260806 11:39:58.747304 6129 fs_manager.cc:359] Metadata directory not provided
I20260806 11:39:58.747390 6129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:39:58.751962 6129 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 11:39:58.754549 6146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:39:58.756107 6129 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 11:39:58.756246 6129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/master-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/master-0/wal
uuid: "62e3c47d9e32427eb798272b95ca13a7"
format_stamp: "Formatted at 2026-08-06 11:39:37 on dist-test-slave-7c35"
server_key: "1e45a4d2d9bff82ef252e9bb15b9e105"
server_key_iv: "04cfce1125d7b46e3c555caf4d012c86"
server_key_version: "encryptionkey@0"
I20260806 11:39:58.756953 6129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:39:58.787389 6129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:39:58.790753 6129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:39:58.791018 6129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:39:58.798902 6129 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.254:36681
I20260806 11:39:58.799234 6199 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.254:36681 every 8 connection(s)
I20260806 11:39:58.800118 6129 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
I20260806 11:39:58.804816 6200 sys_catalog.cc:263] Verifying existing consensus state
I20260806 11:39:58.806592 6200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62e3c47d9e32427eb798272b95ca13a7: Bootstrap starting.
I20260806 11:39:58.808254 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 6129
I20260806 11:39:58.808785 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.193:39037
--local_ip_for_outbound_sockets=127.21.14.193
--tserver_master_addrs=127.21.14.254:36681
--webserver_port=38685
--webserver_interface=127.21.14.193
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab
--principal=kudu/127.21.14.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:43983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 11:39:58.826833 6200 log.cc:824] T 00000000000000000000000000000000 P 62e3c47d9e32427eb798272b95ca13a7: Log is configured to *not* fsync() on all Append() calls
I20260806 11:39:58.836760 6200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62e3c47d9e32427eb798272b95ca13a7: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 11:39:58.837239 6200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62e3c47d9e32427eb798272b95ca13a7: Bootstrap complete.
I20260806 11:39:58.841706 6200 raft_consensus.cc:348] T 00000000000000000000000000000000 P 62e3c47d9e32427eb798272b95ca13a7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62e3c47d9e32427eb798272b95ca13a7" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 36681 } }
I20260806 11:39:58.842146 6200 raft_consensus.cc:729] T 00000000000000000000000000000000 P 62e3c47d9e32427eb798272b95ca13a7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62e3c47d9e32427eb798272b95ca13a7, State: Initialized, Role: FOLLOWER
I20260806 11:39:58.842669 6200 consensus_queue.cc:260] T 00000000000000000000000000000000 P 62e3c47d9e32427eb798272b95ca13a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62e3c47d9e32427eb798272b95ca13a7" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 36681 } }
I20260806 11:39:58.842792 6200 raft_consensus.cc:388] T 00000000000000000000000000000000 P 62e3c47d9e32427eb798272b95ca13a7 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 11:39:58.842911 6200 raft_consensus.cc:482] T 00000000000000000000000000000000 P 62e3c47d9e32427eb798272b95ca13a7 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 11:39:58.843051 6200 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 62e3c47d9e32427eb798272b95ca13a7 [term 1 FOLLOWER]: Advancing to term 2
I20260806 11:39:58.845307 6200 raft_consensus.cc:504] T 00000000000000000000000000000000 P 62e3c47d9e32427eb798272b95ca13a7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62e3c47d9e32427eb798272b95ca13a7" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 36681 } }
I20260806 11:39:58.845696 6200 leader_election.cc:302] T 00000000000000000000000000000000 P 62e3c47d9e32427eb798272b95ca13a7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 62e3c47d9e32427eb798272b95ca13a7; no voters:
I20260806 11:39:58.846074 6200 leader_election.cc:288] T 00000000000000000000000000000000 P 62e3c47d9e32427eb798272b95ca13a7 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 11:39:58.846167 6205 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 62e3c47d9e32427eb798272b95ca13a7 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 11:39:58.846350 6205 raft_consensus.cc:686] T 00000000000000000000000000000000 P 62e3c47d9e32427eb798272b95ca13a7 [term 2 LEADER]: Becoming Leader. State: Replica: 62e3c47d9e32427eb798272b95ca13a7, State: Running, Role: LEADER
I20260806 11:39:58.846689 6205 consensus_queue.cc:237] T 00000000000000000000000000000000 P 62e3c47d9e32427eb798272b95ca13a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62e3c47d9e32427eb798272b95ca13a7" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 36681 } }
I20260806 11:39:58.847525 6200 sys_catalog.cc:564] T 00000000000000000000000000000000 P 62e3c47d9e32427eb798272b95ca13a7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 11:39:58.848381 6206 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62e3c47d9e32427eb798272b95ca13a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "62e3c47d9e32427eb798272b95ca13a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62e3c47d9e32427eb798272b95ca13a7" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 36681 } } }
I20260806 11:39:58.848428 6207 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62e3c47d9e32427eb798272b95ca13a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 62e3c47d9e32427eb798272b95ca13a7. Latest consensus state: current_term: 2 leader_uuid: "62e3c47d9e32427eb798272b95ca13a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62e3c47d9e32427eb798272b95ca13a7" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 36681 } } }
I20260806 11:39:58.848521 6206 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62e3c47d9e32427eb798272b95ca13a7 [sys.catalog]: This master's current role is: LEADER
I20260806 11:39:58.848558 6207 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62e3c47d9e32427eb798272b95ca13a7 [sys.catalog]: This master's current role is: LEADER
I20260806 11:39:58.848951 6214 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 11:39:58.854333 6214 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 11:39:58.855103 6214 catalog_manager.cc:1259] Loaded cluster ID: 1f5eded52c3a49deab2b799491110e1f
I20260806 11:39:58.855173 6214 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 11:39:58.857836 6214 catalog_manager.cc:1504] Loading token signing keys...
I20260806 11:39:58.858662 6214 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 62e3c47d9e32427eb798272b95ca13a7: Loaded TSK: 0
I20260806 11:39:58.880270 6214 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 62e3c47d9e32427eb798272b95ca13a7: Generated new TSK 1
I20260806 11:39:58.880648 6214 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 11:39:58.937402 6203 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:39:58.937754 6203 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:39:58.937850 6203 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:39:58.942247 6203 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:39:58.942363 6203 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:39:58.942461 6203 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.193
I20260806 11:39:58.948468 6203 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:43983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab
--principal=kudu/127.21.14.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.193:39037
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.14.193
--webserver_port=38685
--tserver_master_addrs=127.21.14.254:36681
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6203
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.193
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:39:58.949741 6203 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:39:58.950982 6203 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:39:58.961592 6232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:39:58.961570 6231 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:39:58.961773 6234 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:39:58.962186 6203 server_base.cc:1034] running on GCE node
I20260806 11:39:58.962725 6203 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:39:58.963609 6203 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:39:58.964823 6203 hybrid_clock.cc:648] HybridClock initialized: now 1786016398964777 us; error 54 us; skew 500 ppm
Aug 06 11:39:58 dist-test-slave-7c35 krb5kdc[5543](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016398, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:39:58.968552 6203 init.cc:377] Logged in from keytab as kudu/127.21.14.193@KRBTEST.COM (short username kudu)
I20260806 11:39:58.969770 6203 webserver.cc:460] Webserver started at http://127.21.14.193:38685/ using document root <none> and password file <none>
I20260806 11:39:58.970544 6203 fs_manager.cc:359] Metadata directory not provided
I20260806 11:39:58.970659 6203 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:39:58.976192 6203 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 11:39:58.979286 6241 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:39:58.981020 6203 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 11:39:58.981173 6203 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-0/wal
uuid: "a44ebc4b518a42d5a327cfda159f9635"
format_stamp: "Formatted at 2026-08-06 11:39:37 on dist-test-slave-7c35"
server_key: "f44a7c8cf2c40157cdef5ab6e9d9e01f"
server_key_iv: "400c053b1e170a40f3b2fecf3722e5e7"
server_key_version: "encryptionkey@0"
I20260806 11:39:58.981968 6203 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:39:59.016618 6203 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:39:59.021097 6203 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:39:59.021344 6203 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:39:59.022111 6203 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:39:59.023707 6249 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 11:39:59.028805 6203 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 11:39:59.028934 6203 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260806 11:39:59.029002 6203 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 11:39:59.030967 6203 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 11:39:59.031031 6203 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.002s sys 0.000s
I20260806 11:39:59.031391 6249 tablet_bootstrap.cc:492] T c2a6854e2d0b4037829eb7d29fbd98bc P a44ebc4b518a42d5a327cfda159f9635: Bootstrap starting.
I20260806 11:39:59.043524 6203 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.193:39037
I20260806 11:39:59.043563 6362 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.193:39037 every 8 connection(s)
I20260806 11:39:59.044664 6203 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
I20260806 11:39:59.048003 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 6203
I20260806 11:39:59.048651 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.194:43875
--local_ip_for_outbound_sockets=127.21.14.194
--tserver_master_addrs=127.21.14.254:36681
--webserver_port=36047
--webserver_interface=127.21.14.194
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab
--principal=kudu/127.21.14.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:43983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 11:39:59.053246 6249 log.cc:824] T c2a6854e2d0b4037829eb7d29fbd98bc P a44ebc4b518a42d5a327cfda159f9635: Log is configured to *not* fsync() on all Append() calls
Aug 06 11:39:59 dist-test-slave-7c35 krb5kdc[5543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016398, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.193@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:39:59.062297 6249 tablet_bootstrap.cc:492] T c2a6854e2d0b4037829eb7d29fbd98bc P a44ebc4b518a42d5a327cfda159f9635: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 11:39:59.062760 6249 tablet_bootstrap.cc:492] T c2a6854e2d0b4037829eb7d29fbd98bc P a44ebc4b518a42d5a327cfda159f9635: Bootstrap complete.
I20260806 11:39:59.064335 6249 ts_tablet_manager.cc:1397] T c2a6854e2d0b4037829eb7d29fbd98bc P a44ebc4b518a42d5a327cfda159f9635: Time spent bootstrapping tablet: real 0.033s user 0.017s sys 0.014s
I20260806 11:39:59.067175 6366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:59.047346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:36681 (local address 127.21.14.193:37269)
0806 11:39:59.048633 (+ 1287us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:59.048693 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 11:39:59.049717 (+ 1024us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:39:59.051206 (+ 1489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:39:59.051217 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:39:59.051748 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:39:59.052419 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:59.052442 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:59.053964 (+ 1522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:59.053971 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:59.055520 (+ 1549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:59.055535 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:59.056631 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:59.056638 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:59.056910 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:59.057716 (+ 806us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:39:59.057745 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:39:59.059972 (+ 2227us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:39:59.063231 (+ 3259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:39:59.063240 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:39:59.063258 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:39:59.063650 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:39:59.064166 (+ 516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:39:59.064172 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:39:59.064177 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:39:59.064364 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:39:59.065220 (+ 856us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:39:59.065227 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:39:59.065526 (+ 299us) client_negotiation.cc:769] Sending connection context
0806 11:39:59.065730 (+ 204us) client_negotiation.cc:240] Negotiation successful
0806 11:39:59.066108 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1060,"thread_start_us":160,"threads_started":1}
I20260806 11:39:59.067561 6365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:59.047346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:37269 (local address 127.21.14.254:36681)
0806 11:39:59.048524 (+ 1178us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:59.048549 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:59.049083 (+ 534us) server_negotiation.cc:407] Connection header received
0806 11:39:59.050207 (+ 1124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:59.050219 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:59.050548 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:59.051106 (+ 558us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:39:59.052627 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:59.053772 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:59.055675 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:59.056448 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:59.056621 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:59.060259 (+ 3638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:39:59.060270 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:39:59.060292 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:39:59.060342 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:39:59.063056 (+ 2714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:59.063817 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:59.063826 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:59.063836 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:59.064047 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:59.064542 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:59.064548 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:59.064551 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:59.065080 (+ 529us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:39:59.065221 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:59.065899 (+ 678us) server_negotiation.cc:299] Negotiation successful
0806 11:39:59.066338 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":966,"thread_start_us":159,"threads_started":1}
I20260806 11:39:59.067415 6249 raft_consensus.cc:348] T c2a6854e2d0b4037829eb7d29fbd98bc P a44ebc4b518a42d5a327cfda159f9635 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a44ebc4b518a42d5a327cfda159f9635" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 39037 } } peers { permanent_uuid: "9678ece9548c429ba9076df50c10b444" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 36077 } } peers { permanent_uuid: "558d5fa7ae6d48058c7e74f21bc12211" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 43875 } }
I20260806 11:39:59.067992 6249 raft_consensus.cc:729] T c2a6854e2d0b4037829eb7d29fbd98bc P a44ebc4b518a42d5a327cfda159f9635 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a44ebc4b518a42d5a327cfda159f9635, State: Initialized, Role: FOLLOWER
I20260806 11:39:59.068559 6249 consensus_queue.cc:260] T c2a6854e2d0b4037829eb7d29fbd98bc P a44ebc4b518a42d5a327cfda159f9635 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a44ebc4b518a42d5a327cfda159f9635" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 39037 } } peers { permanent_uuid: "9678ece9548c429ba9076df50c10b444" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 36077 } } peers { permanent_uuid: "558d5fa7ae6d48058c7e74f21bc12211" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 43875 } }
I20260806 11:39:59.069734 6249 ts_tablet_manager.cc:1428] T c2a6854e2d0b4037829eb7d29fbd98bc P a44ebc4b518a42d5a327cfda159f9635: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 11:39:59.070545 6363 heartbeater.cc:347] Connected to a master server at 127.21.14.254:36681
I20260806 11:39:59.070957 6363 heartbeater.cc:464] Registering TS with master...
I20260806 11:39:59.071583 6363 heartbeater.cc:511] Master 127.21.14.254:36681 requested a full tablet report, sending...
I20260806 11:39:59.074141 6163 ts_manager.cc:194] Registered new tserver with Master: a44ebc4b518a42d5a327cfda159f9635 (127.21.14.193:39037)
I20260806 11:39:59.077767 6163 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.14.193@KRBTEST.COM'} at 127.21.14.193:37269
I20260806 11:39:59.080633 6363 heartbeater.cc:503] Master 127.21.14.254:36681 was elected leader, sending a full tablet report...
Aug 06 11:39:59 dist-test-slave-7c35 krb5kdc[5543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016398, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.254@KRBTEST.COM for kudu/127.21.14.193@KRBTEST.COM
I20260806 11:39:59.089243 6371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:59.077210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.193:39037 (local address 127.0.0.1:55234)
0806 11:39:59.077857 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:59.077907 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 11:39:59.078040 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:39:59.078720 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:39:59.078726 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:39:59.079133 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:39:59.079244 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:59.079252 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:59.080236 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:59.080240 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:59.080961 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:59.080971 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:59.081777 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:59.081780 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:59.081909 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:59.082726 (+ 817us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:39:59.082740 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:39:59.084479 (+ 1739us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:39:59.086957 (+ 2478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:39:59.086967 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:39:59.086972 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:39:59.087239 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:39:59.087668 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:39:59.087671 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:39:59.087673 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:39:59.087906 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:39:59.088597 (+ 691us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:39:59.088605 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:39:59.088760 (+ 155us) client_negotiation.cc:769] Sending connection context
0806 11:39:59.088891 (+ 131us) client_negotiation.cc:240] Negotiation successful
0806 11:39:59.089089 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":111,"threads_started":1}
I20260806 11:39:59.089668 6372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:59.077316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55234 (local address 127.21.14.193:39037)
0806 11:39:59.078160 (+ 844us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:59.078173 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:59.078205 (+ 32us) server_negotiation.cc:407] Connection header received
0806 11:39:59.078284 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:59.078289 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:59.078495 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:59.078636 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:39:59.079377 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:59.080108 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:59.081108 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:59.081643 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:59.081756 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:59.084664 (+ 2908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:39:59.084677 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:39:59.084693 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:39:59.084724 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:39:59.086721 (+ 1997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:59.087401 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:59.087408 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:59.087410 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:59.087515 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:59.088045 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:59.088049 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:59.088051 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:59.088418 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:39:59.088559 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:59.089035 (+ 476us) server_negotiation.cc:299] Negotiation successful
0806 11:39:59.089236 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":733,"thread_start_us":121,"threads_started":1}
I20260806 11:39:59.090829 6295 tablet_service.cc:1514] Processing DeleteTablet for tablet c2a6854e2d0b4037829eb7d29fbd98bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:39:38 UTC) from {username='kudu', principal='kudu/127.21.14.254@KRBTEST.COM'} at 127.0.0.1:55234
I20260806 11:39:59.092003 6373 tablet_replica.cc:330] stopping tablet replica
I20260806 11:39:59.092412 6373 raft_consensus.cc:2229] T c2a6854e2d0b4037829eb7d29fbd98bc P a44ebc4b518a42d5a327cfda159f9635 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 11:39:59.092757 6373 raft_consensus.cc:2258] T c2a6854e2d0b4037829eb7d29fbd98bc P a44ebc4b518a42d5a327cfda159f9635 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 11:39:59.094249 6373 ts_tablet_manager.cc:1905] T c2a6854e2d0b4037829eb7d29fbd98bc P a44ebc4b518a42d5a327cfda159f9635: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 11:39:59.100102 6373 ts_tablet_manager.cc:1918] T c2a6854e2d0b4037829eb7d29fbd98bc P a44ebc4b518a42d5a327cfda159f9635: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 11:39:59.100270 6373 log.cc:1196] T c2a6854e2d0b4037829eb7d29fbd98bc P a44ebc4b518a42d5a327cfda159f9635: Deleting WAL directory at /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-0/wal/wals/c2a6854e2d0b4037829eb7d29fbd98bc
I20260806 11:39:59.100572 6373 ts_tablet_manager.cc:1939] T c2a6854e2d0b4037829eb7d29fbd98bc P a44ebc4b518a42d5a327cfda159f9635: Deleting consensus metadata
I20260806 11:39:59.103024 6148 catalog_manager.cc:4841] TS a44ebc4b518a42d5a327cfda159f9635 (127.21.14.193:39037): tablet c2a6854e2d0b4037829eb7d29fbd98bc (table test-table [id=160158dcd525497a920c5d382e47b642]) successfully deleted
W20260806 11:39:59.186515 6367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:39:59.186885 6367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:39:59.186934 6367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:39:59.191502 6367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:39:59.191589 6367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:39:59.191654 6367 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.194
I20260806 11:39:59.197850 6367 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:43983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab
--principal=kudu/127.21.14.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.194:43875
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.14.194
--webserver_port=36047
--tserver_master_addrs=127.21.14.254:36681
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6367
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.194
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:39:59.199175 6367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:39:59.200275 6367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:39:59.211036 6382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:39:59.211036 6380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:39:59.211102 6378 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:39:59.211452 6367 server_base.cc:1034] running on GCE node
I20260806 11:39:59.212036 6367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:39:59.212829 6367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:39:59.214082 6367 hybrid_clock.cc:648] HybridClock initialized: now 1786016399214061 us; error 32 us; skew 500 ppm
Aug 06 11:39:59 dist-test-slave-7c35 krb5kdc[5543](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016399, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:39:59.217609 6367 init.cc:377] Logged in from keytab as kudu/127.21.14.194@KRBTEST.COM (short username kudu)
I20260806 11:39:59.218755 6367 webserver.cc:460] Webserver started at http://127.21.14.194:36047/ using document root <none> and password file <none>
I20260806 11:39:59.219523 6367 fs_manager.cc:359] Metadata directory not provided
I20260806 11:39:59.219619 6367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:39:59.224246 6367 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260806 11:39:59.226755 6389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:39:59.228237 6367 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 11:39:59.228363 6367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-1/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-1/wal
uuid: "558d5fa7ae6d48058c7e74f21bc12211"
format_stamp: "Formatted at 2026-08-06 11:39:37 on dist-test-slave-7c35"
server_key: "dd063a7e8a5d6511a2de98e97fa932ec"
server_key_iv: "deb9c4d2f30c84395fdf40128c335312"
server_key_version: "encryptionkey@0"
I20260806 11:39:59.229023 6367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:39:59.258701 6367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:39:59.262746 6367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:39:59.263053 6367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:39:59.263762 6367 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:39:59.264722 6367 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:39:59.264773 6367 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:39:59.264854 6367 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:39:59.264899 6367 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:39:59.274721 6367 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.194:43875
I20260806 11:39:59.274781 6509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.194:43875 every 8 connection(s)
I20260806 11:39:59.275902 6367 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
I20260806 11:39:59.280584 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 6367
I20260806 11:39:59.281204 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.195:36077
--local_ip_for_outbound_sockets=127.21.14.195
--tserver_master_addrs=127.21.14.254:36681
--webserver_port=32799
--webserver_interface=127.21.14.195
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab
--principal=kudu/127.21.14.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:43983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 11:39:59 dist-test-slave-7c35 krb5kdc[5543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016399, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.194@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:39:59.293802 6365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:59.278008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:56439 (local address 127.21.14.254:36681)
0806 11:39:59.278181 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:59.278185 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:59.279187 (+ 1002us) server_negotiation.cc:407] Connection header received
0806 11:39:59.280935 (+ 1748us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:59.280938 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:59.280997 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:59.281089 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:39:59.282979 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:59.283871 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:59.285139 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:59.285580 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:59.285718 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:59.288922 (+ 3204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:39:59.288929 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:39:59.288935 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:39:59.288986 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:39:59.291020 (+ 2034us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:59.291659 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:59.291665 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:59.291670 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:59.291784 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:59.292266 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:59.292273 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:59.292277 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:59.292588 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:39:59.292716 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:59.293353 (+ 637us) server_negotiation.cc:299] Negotiation successful
0806 11:39:59.293547 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 11:39:59.294936 6512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:59.278423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:36681 (local address 127.21.14.194:56439)
0806 11:39:59.279022 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:59.279071 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 11:39:59.280581 (+ 1510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:39:59.281340 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:39:59.281352 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:39:59.281932 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:39:59.282704 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:59.282728 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:59.284034 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:59.284037 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:59.285002 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:59.285013 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:59.285758 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:59.285760 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:59.285923 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:59.286835 (+ 912us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:39:59.286863 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:39:59.288678 (+ 1815us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:39:59.291181 (+ 2503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:39:59.291188 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:39:59.291205 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:39:59.291535 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:39:59.291941 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:39:59.291943 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:39:59.291945 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:39:59.292144 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:39:59.292733 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:39:59.292739 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:39:59.293059 (+ 320us) client_negotiation.cc:769] Sending connection context
0806 11:39:59.293264 (+ 205us) client_negotiation.cc:240] Negotiation successful
0806 11:39:59.293726 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":145,"threads_started":1}
I20260806 11:39:59.296591 6510 heartbeater.cc:347] Connected to a master server at 127.21.14.254:36681
I20260806 11:39:59.296871 6510 heartbeater.cc:464] Registering TS with master...
I20260806 11:39:59.297550 6510 heartbeater.cc:511] Master 127.21.14.254:36681 requested a full tablet report, sending...
I20260806 11:39:59.298771 6163 ts_manager.cc:194] Registered new tserver with Master: 558d5fa7ae6d48058c7e74f21bc12211 (127.21.14.194:43875)
I20260806 11:39:59.299602 6163 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.14.194@KRBTEST.COM'} at 127.21.14.194:56439
W20260806 11:39:59.408041 6513 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:39:59.408327 6513 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:39:59.408375 6513 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:39:59.412695 6513 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:39:59.412763 6513 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:39:59.412824 6513 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.195
I20260806 11:39:59.418258 6513 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:43983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab
--principal=kudu/127.21.14.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.195:36077
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.14.195
--webserver_port=32799
--tserver_master_addrs=127.21.14.254:36681
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6513
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.195
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:39:59.419548 6513 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:39:59.420643 6513 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:39:59.431785 6520 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:39:59.431785 6523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:39:59.431806 6519 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:39:59.432400 6513 server_base.cc:1034] running on GCE node
I20260806 11:39:59.432946 6513 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:39:59.433732 6513 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:39:59.435009 6513 hybrid_clock.cc:648] HybridClock initialized: now 1786016399434976 us; error 32 us; skew 500 ppm
Aug 06 11:39:59 dist-test-slave-7c35 krb5kdc[5543](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016399, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:39:59.438140 6513 init.cc:377] Logged in from keytab as kudu/127.21.14.195@KRBTEST.COM (short username kudu)
I20260806 11:39:59.439198 6513 webserver.cc:460] Webserver started at http://127.21.14.195:32799/ using document root <none> and password file <none>
I20260806 11:39:59.439811 6513 fs_manager.cc:359] Metadata directory not provided
I20260806 11:39:59.439893 6513 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:39:59.444344 6513 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260806 11:39:59.446776 6530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:39:59.448467 6513 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 11:39:59.448661 6513 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-2/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-2/wal
uuid: "9678ece9548c429ba9076df50c10b444"
format_stamp: "Formatted at 2026-08-06 11:39:37 on dist-test-slave-7c35"
server_key: "6b73a94d75cc4e1fa264b6b69f5d24fe"
server_key_iv: "494109ca392fea40387785f02eebeb71"
server_key_version: "encryptionkey@0"
I20260806 11:39:59.449343 6513 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:39:59.481407 6513 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:39:59.485041 6513 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:39:59.485291 6513 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:39:59.485996 6513 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:39:59.487033 6513 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:39:59.487087 6513 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:39:59.487174 6513 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:39:59.487215 6513 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:39:59.498028 6513 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.195:36077
I20260806 11:39:59.498063 6649 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.195:36077 every 8 connection(s)
I20260806 11:39:59.499330 6513 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
I20260806 11:39:59.500967 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 6513
Aug 06 11:39:59 dist-test-slave-7c35 krb5kdc[5543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016399, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.195@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:39:59.512235 6652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:59.501726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56284 (local address 127.21.14.254:36681)
0806 11:39:59.501968 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:59.501973 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:59.503211 (+ 1238us) server_negotiation.cc:407] Connection header received
0806 11:39:59.503302 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:59.503304 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:59.503365 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:59.503486 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:39:59.504290 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:59.505127 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:59.506229 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:59.506471 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:59.506559 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:59.508466 (+ 1907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:39:59.508470 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:39:59.508473 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:39:59.508502 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:39:59.510278 (+ 1776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:59.510826 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:59.510830 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:59.510834 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:59.510967 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:59.511290 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:59.511293 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:59.511294 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:59.511469 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:39:59.511697 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:59.511902 (+ 205us) server_negotiation.cc:299] Negotiation successful
0806 11:39:59.512055 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":183,"thread_start_us":83,"threads_started":1}
I20260806 11:39:59.515744 6365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:59.501570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:59597 (local address 127.21.14.254:36681)
0806 11:39:59.501732 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:59.501738 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:59.502799 (+ 1061us) server_negotiation.cc:407] Connection header received
0806 11:39:59.504391 (+ 1592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:59.504394 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:59.504439 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:59.504501 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:39:59.505874 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:59.506597 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:59.507835 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:59.508274 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:59.508400 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:59.511554 (+ 3154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:39:59.511557 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:39:59.511559 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:39:59.511581 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:39:59.513239 (+ 1658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:59.514016 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:59.514019 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:59.514020 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:59.514095 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:59.514565 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:59.514569 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:59.514571 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:59.514792 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:39:59.514935 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:59.515496 (+ 561us) server_negotiation.cc:299] Negotiation successful
0806 11:39:59.515622 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 11:39:59.516669 6654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:59.502136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:36681 (local address 127.21.14.195:59597)
0806 11:39:59.502632 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:59.502679 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 11:39:59.504163 (+ 1484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:39:59.504703 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:39:59.504713 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:39:59.505177 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:39:59.505720 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:59.505734 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:59.506737 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:59.506739 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:59.507690 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:59.507699 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:59.508489 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:59.508523 (+ 34us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:59.508760 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:59.509271 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:39:59.509292 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:39:59.511344 (+ 2052us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:39:59.513455 (+ 2111us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:39:59.513461 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:39:59.513476 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:39:59.513903 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:39:59.514193 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:39:59.514197 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:39:59.514199 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:39:59.514407 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:39:59.514954 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:39:59.514959 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:39:59.515232 (+ 273us) client_negotiation.cc:769] Sending connection context
0806 11:39:59.515398 (+ 166us) client_negotiation.cc:240] Negotiation successful
0806 11:39:59.515713 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":128,"threads_started":1}
I20260806 11:39:59.517884 6650 heartbeater.cc:347] Connected to a master server at 127.21.14.254:36681
I20260806 11:39:59.518170 6650 heartbeater.cc:464] Registering TS with master...
I20260806 11:39:59.518741 6650 heartbeater.cc:511] Master 127.21.14.254:36681 requested a full tablet report, sending...
I20260806 11:39:59.519950 6163 ts_manager.cc:194] Registered new tserver with Master: 9678ece9548c429ba9076df50c10b444 (127.21.14.195:36077)
I20260806 11:39:59.520586 6163 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.14.195@KRBTEST.COM'} at 127.21.14.195:59597
I20260806 11:39:59.526752 21563 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 11:39:59.531052 6365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:59.527459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56298 (local address 127.21.14.254:36681)
0806 11:39:59.527594 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:59.527598 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:59.527788 (+ 190us) server_negotiation.cc:407] Connection header received
0806 11:39:59.527924 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:59.527926 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:59.528039 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:59.528131 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 11:39:59.528388 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:59.528923 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:59.529811 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:59.530042 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:59.530172 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:59.530467 (+ 295us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 11:39:59.530620 (+ 153us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0806 11:39:59.530919 (+ 299us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:56298: token expired
Metrics: {"server-negotiator.queue_time_us":48}
W20260806 11:39:59.531131 6365 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:56298: token expired
W20260806 11:39:59.531466 6083 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.21.14.254:36681)
I20260806 11:39:59.540076 6365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:59.531894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56314 (local address 127.21.14.254:36681)
0806 11:39:59.532040 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:59.532047 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:59.532106 (+ 59us) server_negotiation.cc:407] Connection header received
0806 11:39:59.532270 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:59.532276 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:59.532326 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:59.532459 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:39:59.533300 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:59.533856 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:59.534745 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:59.535093 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:59.535219 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:59.536801 (+ 1582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:39:59.536805 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:39:59.536808 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:39:59.536842 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:39:59.538268 (+ 1426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:59.538730 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:59.538735 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:59.538737 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:59.538827 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:39:59.539164 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:39:59.539168 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:39:59.539170 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:39:59.539369 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:39:59.539517 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:59.539766 (+ 249us) server_negotiation.cc:299] Negotiation successful
0806 11:39:59.539888 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 11:39:59.542832 6163 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56314:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 11:39:59.546429 6163 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 11:39:59.555447 6295 tablet_service.cc:1467] Processing CreateTablet for tablet 69fef14c78034517899675c07573120e (DEFAULT_TABLE table=test-table [id=1e7fa0fac75747038d1cbc04f5387fcd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:39:59.556202 6295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69fef14c78034517899675c07573120e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:39:59.559165 6249 tablet_bootstrap.cc:492] T 69fef14c78034517899675c07573120e P a44ebc4b518a42d5a327cfda159f9635: Bootstrap starting.
I20260806 11:39:59.559674 6371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:59.554263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.194:43875 (local address 127.0.0.1:38430)
0806 11:39:59.554394 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:59.554411 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 11:39:59.554525 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:39:59.555459 (+ 934us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:39:59.555462 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:39:59.555485 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:39:59.555604 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:59.555611 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:59.556965 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:59.556970 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:59.557789 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:59.557798 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:59.559209 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:59.559213 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:59.559327 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:59.559394 (+ 67us) client_negotiation.cc:769] Sending connection context
0806 11:39:59.559447 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 11:39:59.559551 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260806 11:39:59.560129 6249 tablet_bootstrap.cc:654] T 69fef14c78034517899675c07573120e P a44ebc4b518a42d5a327cfda159f9635: Neither blocks nor log segments found. Creating new log.
I20260806 11:39:59.560508 6659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:59.554453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38430 (local address 127.21.14.194:43875)
0806 11:39:59.554889 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:59.554904 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:59.554932 (+ 28us) server_negotiation.cc:407] Connection header received
0806 11:39:59.555024 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:59.555030 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:59.555270 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:59.555454 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:39:59.555713 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:59.556843 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:59.558160 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:59.559066 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:59.560050 (+ 984us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:59.560205 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:59.560266 (+ 61us) server_negotiation.cc:299] Negotiation successful
0806 11:39:59.560382 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":151,"threads_started":1}
I20260806 11:39:59.560786 6660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:59.554716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.195:36077 (local address 127.0.0.1:54304)
0806 11:39:59.555033 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:59.555048 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 11:39:59.555139 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:39:59.556389 (+ 1250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:39:59.556392 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:39:59.556422 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:39:59.556519 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:59.556526 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:59.558055 (+ 1529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:59.558058 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:59.559216 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:59.559224 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:59.560330 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:59.560335 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:59.560448 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:59.560482 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 11:39:59.560550 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 11:39:59.560645 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":137,"threads_started":1}
I20260806 11:39:59.561458 6661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:59.555167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54304 (local address 127.21.14.195:36077)
0806 11:39:59.555912 (+ 745us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:59.555924 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:59.555948 (+ 24us) server_negotiation.cc:407] Connection header received
0806 11:39:59.556024 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:59.556028 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:59.556220 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:59.556674 (+ 454us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:39:59.556862 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:59.557932 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:59.559315 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:59.560151 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:59.560973 (+ 822us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:59.561134 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:59.561187 (+ 53us) server_negotiation.cc:299] Negotiation successful
0806 11:39:59.561296 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":625,"thread_start_us":189,"threads_started":1}
I20260806 11:39:59.561909 6249 tablet_bootstrap.cc:492] T 69fef14c78034517899675c07573120e P a44ebc4b518a42d5a327cfda159f9635: No bootstrap required, opened a new log
I20260806 11:39:59.561977 6249 ts_tablet_manager.cc:1397] T 69fef14c78034517899675c07573120e P a44ebc4b518a42d5a327cfda159f9635: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 11:39:59.562454 6249 raft_consensus.cc:348] T 69fef14c78034517899675c07573120e P a44ebc4b518a42d5a327cfda159f9635 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558d5fa7ae6d48058c7e74f21bc12211" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 43875 } } peers { permanent_uuid: "9678ece9548c429ba9076df50c10b444" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 36077 } } peers { permanent_uuid: "a44ebc4b518a42d5a327cfda159f9635" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 39037 } }
I20260806 11:39:59.562546 6249 raft_consensus.cc:374] T 69fef14c78034517899675c07573120e P a44ebc4b518a42d5a327cfda159f9635 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:39:59.562572 6249 raft_consensus.cc:729] T 69fef14c78034517899675c07573120e P a44ebc4b518a42d5a327cfda159f9635 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a44ebc4b518a42d5a327cfda159f9635, State: Initialized, Role: FOLLOWER
I20260806 11:39:59.562713 6249 consensus_queue.cc:260] T 69fef14c78034517899675c07573120e P a44ebc4b518a42d5a327cfda159f9635 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558d5fa7ae6d48058c7e74f21bc12211" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 43875 } } peers { permanent_uuid: "9678ece9548c429ba9076df50c10b444" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 36077 } } peers { permanent_uuid: "a44ebc4b518a42d5a327cfda159f9635" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 39037 } }
I20260806 11:39:59.563038 6249 ts_tablet_manager.cc:1428] T 69fef14c78034517899675c07573120e P a44ebc4b518a42d5a327cfda159f9635: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 11:39:59.563675 6440 tablet_service.cc:1467] Processing CreateTablet for tablet 69fef14c78034517899675c07573120e (DEFAULT_TABLE table=test-table [id=1e7fa0fac75747038d1cbc04f5387fcd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:39:59.564733 6440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69fef14c78034517899675c07573120e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:39:59.569815 6581 tablet_service.cc:1467] Processing CreateTablet for tablet 69fef14c78034517899675c07573120e (DEFAULT_TABLE table=test-table [id=1e7fa0fac75747038d1cbc04f5387fcd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:39:59.571146 6581 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69fef14c78034517899675c07573120e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:39:59.572628 6663 tablet_bootstrap.cc:492] T 69fef14c78034517899675c07573120e P 558d5fa7ae6d48058c7e74f21bc12211: Bootstrap starting.
I20260806 11:39:59.575404 6663 tablet_bootstrap.cc:654] T 69fef14c78034517899675c07573120e P 558d5fa7ae6d48058c7e74f21bc12211: Neither blocks nor log segments found. Creating new log.
I20260806 11:39:59.576190 6663 log.cc:824] T 69fef14c78034517899675c07573120e P 558d5fa7ae6d48058c7e74f21bc12211: Log is configured to *not* fsync() on all Append() calls
I20260806 11:39:59.578317 6663 tablet_bootstrap.cc:492] T 69fef14c78034517899675c07573120e P 558d5fa7ae6d48058c7e74f21bc12211: No bootstrap required, opened a new log
I20260806 11:39:59.578504 6666 tablet_bootstrap.cc:492] T 69fef14c78034517899675c07573120e P 9678ece9548c429ba9076df50c10b444: Bootstrap starting.
I20260806 11:39:59.578560 6663 ts_tablet_manager.cc:1397] T 69fef14c78034517899675c07573120e P 558d5fa7ae6d48058c7e74f21bc12211: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 11:39:59.581053 6666 tablet_bootstrap.cc:654] T 69fef14c78034517899675c07573120e P 9678ece9548c429ba9076df50c10b444: Neither blocks nor log segments found. Creating new log.
I20260806 11:39:59.582557 6666 log.cc:824] T 69fef14c78034517899675c07573120e P 9678ece9548c429ba9076df50c10b444: Log is configured to *not* fsync() on all Append() calls
I20260806 11:39:59.582913 6663 raft_consensus.cc:348] T 69fef14c78034517899675c07573120e P 558d5fa7ae6d48058c7e74f21bc12211 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558d5fa7ae6d48058c7e74f21bc12211" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 43875 } } peers { permanent_uuid: "9678ece9548c429ba9076df50c10b444" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 36077 } } peers { permanent_uuid: "a44ebc4b518a42d5a327cfda159f9635" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 39037 } }
I20260806 11:39:59.583279 6663 raft_consensus.cc:374] T 69fef14c78034517899675c07573120e P 558d5fa7ae6d48058c7e74f21bc12211 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:39:59.583348 6663 raft_consensus.cc:729] T 69fef14c78034517899675c07573120e P 558d5fa7ae6d48058c7e74f21bc12211 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558d5fa7ae6d48058c7e74f21bc12211, State: Initialized, Role: FOLLOWER
I20260806 11:39:59.583892 6663 consensus_queue.cc:260] T 69fef14c78034517899675c07573120e P 558d5fa7ae6d48058c7e74f21bc12211 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558d5fa7ae6d48058c7e74f21bc12211" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 43875 } } peers { permanent_uuid: "9678ece9548c429ba9076df50c10b444" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 36077 } } peers { permanent_uuid: "a44ebc4b518a42d5a327cfda159f9635" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 39037 } }
I20260806 11:39:59.584702 6510 heartbeater.cc:503] Master 127.21.14.254:36681 was elected leader, sending a full tablet report...
I20260806 11:39:59.585021 6663 ts_tablet_manager.cc:1428] T 69fef14c78034517899675c07573120e P 558d5fa7ae6d48058c7e74f21bc12211: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 11:39:59.585134 6666 tablet_bootstrap.cc:492] T 69fef14c78034517899675c07573120e P 9678ece9548c429ba9076df50c10b444: No bootstrap required, opened a new log
I20260806 11:39:59.585348 6666 ts_tablet_manager.cc:1397] T 69fef14c78034517899675c07573120e P 9678ece9548c429ba9076df50c10b444: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 11:39:59.589048 6666 raft_consensus.cc:348] T 69fef14c78034517899675c07573120e P 9678ece9548c429ba9076df50c10b444 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558d5fa7ae6d48058c7e74f21bc12211" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 43875 } } peers { permanent_uuid: "9678ece9548c429ba9076df50c10b444" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 36077 } } peers { permanent_uuid: "a44ebc4b518a42d5a327cfda159f9635" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 39037 } }
I20260806 11:39:59.589326 6666 raft_consensus.cc:374] T 69fef14c78034517899675c07573120e P 9678ece9548c429ba9076df50c10b444 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:39:59.589385 6666 raft_consensus.cc:729] T 69fef14c78034517899675c07573120e P 9678ece9548c429ba9076df50c10b444 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9678ece9548c429ba9076df50c10b444, State: Initialized, Role: FOLLOWER
I20260806 11:39:59.589813 6666 consensus_queue.cc:260] T 69fef14c78034517899675c07573120e P 9678ece9548c429ba9076df50c10b444 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558d5fa7ae6d48058c7e74f21bc12211" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 43875 } } peers { permanent_uuid: "9678ece9548c429ba9076df50c10b444" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 36077 } } peers { permanent_uuid: "a44ebc4b518a42d5a327cfda159f9635" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 39037 } }
I20260806 11:39:59.590629 6650 heartbeater.cc:503] Master 127.21.14.254:36681 was elected leader, sending a full tablet report...
I20260806 11:39:59.590986 6666 ts_tablet_manager.cc:1428] T 69fef14c78034517899675c07573120e P 9678ece9548c429ba9076df50c10b444: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
W20260806 11:39:59.750545 6651 tablet.cc:2369] T 69fef14c78034517899675c07573120e P 9678ece9548c429ba9076df50c10b444: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 11:39:59.777354 6511 tablet.cc:2369] T 69fef14c78034517899675c07573120e P 558d5fa7ae6d48058c7e74f21bc12211: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 11:39:59.796314 6364 tablet.cc:2369] T 69fef14c78034517899675c07573120e P a44ebc4b518a42d5a327cfda159f9635: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 11:39:59.850723 6669 raft_consensus.cc:482] T 69fef14c78034517899675c07573120e P 9678ece9548c429ba9076df50c10b444 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 11:39:59.851063 6669 raft_consensus.cc:504] T 69fef14c78034517899675c07573120e P 9678ece9548c429ba9076df50c10b444 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558d5fa7ae6d48058c7e74f21bc12211" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 43875 } } peers { permanent_uuid: "9678ece9548c429ba9076df50c10b444" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 36077 } } peers { permanent_uuid: "a44ebc4b518a42d5a327cfda159f9635" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 39037 } }
I20260806 11:39:59.852401 6669 leader_election.cc:288] T 69fef14c78034517899675c07573120e P 9678ece9548c429ba9076df50c10b444 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 558d5fa7ae6d48058c7e74f21bc12211 (127.21.14.194:43875), a44ebc4b518a42d5a327cfda159f9635 (127.21.14.193:39037)
I20260806 11:39:59.857554 6654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:59.852736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.193:39037 (local address 127.21.14.195:50839)
0806 11:39:59.852899 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:59.852919 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 11:39:59.853047 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:39:59.853683 (+ 636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:39:59.853687 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:39:59.853708 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:39:59.853850 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:59.853857 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:59.854900 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:59.854906 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:59.855865 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:59.855876 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:59.857014 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:59.857021 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:59.857155 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:59.857189 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 11:39:59.857270 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 11:39:59.857374 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62,"mutex_wait_us":1}
I20260806 11:39:59.857587 6670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:59.852850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:50839 (local address 127.21.14.193:39037)
0806 11:39:59.853249 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:59.853255 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:59.853277 (+ 22us) server_negotiation.cc:407] Connection header received
0806 11:39:59.853344 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:59.853348 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:59.853430 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:59.853682 (+ 252us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:39:59.854048 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:59.854728 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:59.856007 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:59.856814 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:59.856999 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:59.857204 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:59.857306 (+ 102us) server_negotiation.cc:299] Negotiation successful
0806 11:39:59.857404 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":121,"threads_started":1}
I20260806 11:39:59.857607 6659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:59.852979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:38067 (local address 127.21.14.194:43875)
0806 11:39:59.853202 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:59.853206 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:59.853616 (+ 410us) server_negotiation.cc:407] Connection header received
0806 11:39:59.853792 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:59.853795 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:59.853856 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:59.853954 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:39:59.854243 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:59.854995 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:59.856334 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:59.856992 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:59.857124 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:59.857174 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:59.857400 (+ 226us) server_negotiation.cc:299] Negotiation successful
0806 11:39:59.857490 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
I20260806 11:39:59.857563 6671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:59.852736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.194:43875 (local address 127.21.14.195:38067)
0806 11:39:59.853479 (+ 743us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:59.853518 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 11:39:59.853664 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:39:59.853966 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:39:59.853970 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:39:59.853986 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:39:59.854121 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:59.854127 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:59.855115 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:59.855122 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:59.856153 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:39:59.856163 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:39:59.857106 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:39:59.857113 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:39:59.857203 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:59.857232 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 11:39:59.857352 (+ 120us) client_negotiation.cc:240] Negotiation successful
0806 11:39:59.857419 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":681,"mutex_wait_us":356,"thread_start_us":119,"threads_started":1}
I20260806 11:39:59.858371 6316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69fef14c78034517899675c07573120e" candidate_uuid: "9678ece9548c429ba9076df50c10b444" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a44ebc4b518a42d5a327cfda159f9635" is_pre_election: true
I20260806 11:39:59.858706 6316 raft_consensus.cc:2454] T 69fef14c78034517899675c07573120e P a44ebc4b518a42d5a327cfda159f9635 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9678ece9548c429ba9076df50c10b444 in term 0.
I20260806 11:39:59.859184 6462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69fef14c78034517899675c07573120e" candidate_uuid: "9678ece9548c429ba9076df50c10b444" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558d5fa7ae6d48058c7e74f21bc12211" is_pre_election: true
I20260806 11:39:59.859297 6532 leader_election.cc:302] T 69fef14c78034517899675c07573120e P 9678ece9548c429ba9076df50c10b444 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9678ece9548c429ba9076df50c10b444, a44ebc4b518a42d5a327cfda159f9635; no voters:
I20260806 11:39:59.859534 6462 raft_consensus.cc:2454] T 69fef14c78034517899675c07573120e P 558d5fa7ae6d48058c7e74f21bc12211 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9678ece9548c429ba9076df50c10b444 in term 0.
I20260806 11:39:59.859670 6669 raft_consensus.cc:2781] T 69fef14c78034517899675c07573120e P 9678ece9548c429ba9076df50c10b444 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 11:39:59.859808 6669 raft_consensus.cc:482] T 69fef14c78034517899675c07573120e P 9678ece9548c429ba9076df50c10b444 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 11:39:59.859879 6669 raft_consensus.cc:3037] T 69fef14c78034517899675c07573120e P 9678ece9548c429ba9076df50c10b444 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:39:59.861141 6669 raft_consensus.cc:504] T 69fef14c78034517899675c07573120e P 9678ece9548c429ba9076df50c10b444 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558d5fa7ae6d48058c7e74f21bc12211" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 43875 } } peers { permanent_uuid: "9678ece9548c429ba9076df50c10b444" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 36077 } } peers { permanent_uuid: "a44ebc4b518a42d5a327cfda159f9635" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 39037 } }
I20260806 11:39:59.861567 6669 leader_election.cc:288] T 69fef14c78034517899675c07573120e P 9678ece9548c429ba9076df50c10b444 [CANDIDATE]: Term 1 election: Requested vote from peers 558d5fa7ae6d48058c7e74f21bc12211 (127.21.14.194:43875), a44ebc4b518a42d5a327cfda159f9635 (127.21.14.193:39037)
I20260806 11:39:59.862008 6316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69fef14c78034517899675c07573120e" candidate_uuid: "9678ece9548c429ba9076df50c10b444" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a44ebc4b518a42d5a327cfda159f9635"
I20260806 11:39:59.862193 6316 raft_consensus.cc:3037] T 69fef14c78034517899675c07573120e P a44ebc4b518a42d5a327cfda159f9635 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:39:59.863520 6316 raft_consensus.cc:2454] T 69fef14c78034517899675c07573120e P a44ebc4b518a42d5a327cfda159f9635 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9678ece9548c429ba9076df50c10b444 in term 1.
I20260806 11:39:59.863904 6532 leader_election.cc:302] T 69fef14c78034517899675c07573120e P 9678ece9548c429ba9076df50c10b444 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9678ece9548c429ba9076df50c10b444, a44ebc4b518a42d5a327cfda159f9635; no voters:
I20260806 11:39:59.863888 6462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69fef14c78034517899675c07573120e" candidate_uuid: "9678ece9548c429ba9076df50c10b444" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558d5fa7ae6d48058c7e74f21bc12211"
I20260806 11:39:59.864040 6462 raft_consensus.cc:3037] T 69fef14c78034517899675c07573120e P 558d5fa7ae6d48058c7e74f21bc12211 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:39:59.864117 6669 raft_consensus.cc:2781] T 69fef14c78034517899675c07573120e P 9678ece9548c429ba9076df50c10b444 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:39:59.864396 6669 raft_consensus.cc:686] T 69fef14c78034517899675c07573120e P 9678ece9548c429ba9076df50c10b444 [term 1 LEADER]: Becoming Leader. State: Replica: 9678ece9548c429ba9076df50c10b444, State: Running, Role: LEADER
I20260806 11:39:59.864817 6669 consensus_queue.cc:237] T 69fef14c78034517899675c07573120e P 9678ece9548c429ba9076df50c10b444 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558d5fa7ae6d48058c7e74f21bc12211" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 43875 } } peers { permanent_uuid: "9678ece9548c429ba9076df50c10b444" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 36077 } } peers { permanent_uuid: "a44ebc4b518a42d5a327cfda159f9635" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 39037 } }
I20260806 11:39:59.865547 6462 raft_consensus.cc:2454] T 69fef14c78034517899675c07573120e P 558d5fa7ae6d48058c7e74f21bc12211 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9678ece9548c429ba9076df50c10b444 in term 1.
I20260806 11:39:59.868721 6163 catalog_manager.cc:5495] T 69fef14c78034517899675c07573120e P 9678ece9548c429ba9076df50c10b444 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9678ece9548c429ba9076df50c10b444 (127.21.14.195). New cstate: current_term: 1 leader_uuid: "9678ece9548c429ba9076df50c10b444" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558d5fa7ae6d48058c7e74f21bc12211" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 43875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9678ece9548c429ba9076df50c10b444" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 36077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a44ebc4b518a42d5a327cfda159f9635" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 39037 } health_report { overall_health: UNKNOWN } } }
I20260806 11:39:59.936483 6661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:59.932646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54316 (local address 127.21.14.195:36077)
0806 11:39:59.932784 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 11:39:59.932788 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:39:59.932804 (+ 16us) server_negotiation.cc:407] Connection header received
0806 11:39:59.932901 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:39:59.932903 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:39:59.932960 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:39:59.933123 (+ 163us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 11:39:59.933388 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:59.933930 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:59.934904 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:39:59.935112 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:39:59.935194 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:39:59.935526 (+ 332us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 11:39:59.935983 (+ 457us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 11:39:59.936085 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 11:39:59.936225 (+ 140us) server_negotiation.cc:299] Negotiation successful
0806 11:39:59.936282 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 11:39:59.943128 6316 raft_consensus.cc:1261] T 69fef14c78034517899675c07573120e P a44ebc4b518a42d5a327cfda159f9635 [term 1 FOLLOWER]: Refusing update from remote peer 9678ece9548c429ba9076df50c10b444: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 11:39:59.943145 6462 raft_consensus.cc:1261] T 69fef14c78034517899675c07573120e P 558d5fa7ae6d48058c7e74f21bc12211 [term 1 FOLLOWER]: Refusing update from remote peer 9678ece9548c429ba9076df50c10b444: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 11:39:59.943910 6672 consensus_queue.cc:1035] T 69fef14c78034517899675c07573120e P 9678ece9548c429ba9076df50c10b444 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a44ebc4b518a42d5a327cfda159f9635" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 39037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 11:39:59.944113 6669 consensus_queue.cc:1035] T 69fef14c78034517899675c07573120e P 9678ece9548c429ba9076df50c10b444 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558d5fa7ae6d48058c7e74f21bc12211" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 43875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 11:39:59.953889 6676 mvcc.cc:205] Tried to move back new op lower bound from 7315523174158598144 to 7315523173850202112. Current Snapshot: MvccSnapshot[applied={T|T < 7315523174158598144}]
I20260806 11:39:59.960186 6162 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56314:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 11:39:59.960451 6162 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56314:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 11:39:59.963478 6162 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 62e3c47d9e32427eb798272b95ca13a7: Sending DeleteTablet for 3 replicas of tablet 69fef14c78034517899675c07573120e
I20260806 11:39:59.964489 6440 tablet_service.cc:1514] Processing DeleteTablet for tablet 69fef14c78034517899675c07573120e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:39:59 UTC) from {username='kudu', principal='kudu/127.21.14.254@KRBTEST.COM'} at 127.0.0.1:38430
I20260806 11:39:59.964526 6581 tablet_service.cc:1514] Processing DeleteTablet for tablet 69fef14c78034517899675c07573120e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:39:59 UTC) from {username='kudu', principal='kudu/127.21.14.254@KRBTEST.COM'} at 127.0.0.1:54304
I20260806 11:39:59.964744 6295 tablet_service.cc:1514] Processing DeleteTablet for tablet 69fef14c78034517899675c07573120e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:39:59 UTC) from {username='kudu', principal='kudu/127.21.14.254@KRBTEST.COM'} at 127.0.0.1:55234
I20260806 11:39:59.965165 6685 tablet_replica.cc:330] T 69fef14c78034517899675c07573120e P 9678ece9548c429ba9076df50c10b444: stopping tablet replica
I20260806 11:39:59.965413 6686 tablet_replica.cc:330] T 69fef14c78034517899675c07573120e P a44ebc4b518a42d5a327cfda159f9635: stopping tablet replica
I20260806 11:39:59.965430 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 6203
I20260806 11:39:59.968344 6684 tablet_replica.cc:330] T 69fef14c78034517899675c07573120e P 558d5fa7ae6d48058c7e74f21bc12211: stopping tablet replica
I20260806 11:39:59.969059 6684 raft_consensus.cc:2229] T 69fef14c78034517899675c07573120e P 558d5fa7ae6d48058c7e74f21bc12211 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 11:39:59.969467 6684 raft_consensus.cc:2258] T 69fef14c78034517899675c07573120e P 558d5fa7ae6d48058c7e74f21bc12211 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 11:39:59.972429 6684 ts_tablet_manager.cc:1905] T 69fef14c78034517899675c07573120e P 558d5fa7ae6d48058c7e74f21bc12211: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 11:39:59.973878 6685 raft_consensus.cc:2229] T 69fef14c78034517899675c07573120e P 9678ece9548c429ba9076df50c10b444 [term 1 LEADER]: Raft consensus shutting down.
I20260806 11:39:59.974395 6685 raft_consensus.cc:2258] T 69fef14c78034517899675c07573120e P 9678ece9548c429ba9076df50c10b444 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 11:39:59.976253 6685 ts_tablet_manager.cc:1905] T 69fef14c78034517899675c07573120e P 9678ece9548c429ba9076df50c10b444: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 11:39:59.976815 6684 ts_tablet_manager.cc:1918] T 69fef14c78034517899675c07573120e P 558d5fa7ae6d48058c7e74f21bc12211: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 11:39:59.976969 6684 log.cc:1196] T 69fef14c78034517899675c07573120e P 558d5fa7ae6d48058c7e74f21bc12211: Deleting WAL directory at /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-1/wal/wals/69fef14c78034517899675c07573120e
I20260806 11:39:59.977378 6684 ts_tablet_manager.cc:1939] T 69fef14c78034517899675c07573120e P 558d5fa7ae6d48058c7e74f21bc12211: Deleting consensus metadata
I20260806 11:39:59.979151 6147 catalog_manager.cc:4841] TS 558d5fa7ae6d48058c7e74f21bc12211 (127.21.14.194:43875): tablet 69fef14c78034517899675c07573120e (table test-table [id=1e7fa0fac75747038d1cbc04f5387fcd]) successfully deleted
I20260806 11:39:59.979532 6685 ts_tablet_manager.cc:1918] T 69fef14c78034517899675c07573120e P 9678ece9548c429ba9076df50c10b444: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 11:39:59.979642 6685 log.cc:1196] T 69fef14c78034517899675c07573120e P 9678ece9548c429ba9076df50c10b444: Deleting WAL directory at /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-2/wal/wals/69fef14c78034517899675c07573120e
I20260806 11:39:59.979980 6685 ts_tablet_manager.cc:1939] T 69fef14c78034517899675c07573120e P 9678ece9548c429ba9076df50c10b444: Deleting consensus metadata
I20260806 11:39:59.981312 6150 catalog_manager.cc:4841] TS 9678ece9548c429ba9076df50c10b444 (127.21.14.195:36077): tablet 69fef14c78034517899675c07573120e (table test-table [id=1e7fa0fac75747038d1cbc04f5387fcd]) successfully deleted
W20260806 11:39:59.982275 6148 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.14.193:39037 (error 108)
I20260806 11:39:59.983070 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 6367
I20260806 11:39:59.984679 6660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:39:59.983214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.193:39037 (local address 127.0.0.1:55248)
0806 11:39:59.984503 (+ 1289us) negotiation.cc:107] Waiting for socket to connect
0806 11:39:59.984585 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.193:39037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1182}
W20260806 11:39:59.987421 6148 catalog_manager.cc:4584] TS a44ebc4b518a42d5a327cfda159f9635 (127.21.14.193:39037): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 69fef14c78034517899675c07573120e: Network error: Client connection negotiation failed: client connection to 127.21.14.193:39037: connect: Connection refused (error 111)
I20260806 11:39:59.993600 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 6513
I20260806 11:40:00.002205 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 6129
I20260806 11:40:20.012377 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:36681
--webserver_interface=127.21.14.254
--webserver_port=36065
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab
--principal=kudu/127.21.14.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:43983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.14.254:36681
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:40:20.136746 6688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:40:20.137269 6688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:40:20.137367 6688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:40:20.143203 6688 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 11:40:20.143289 6688 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 11:40:20.143307 6688 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 11:40:20.143321 6688 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 11:40:20.143342 6688 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:40:20.143361 6688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:40:20.143378 6688 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 11:40:20.143394 6688 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 11:40:20.149333 6688 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:43983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.21.14.254:36681
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab
--principal=kudu/127.21.14.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:36681
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.14.254
--webserver_port=36065
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6688
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:40:20.150599 6688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:40:20.151690 6688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:40:20.162561 6693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:40:20.162686 6697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:40:20.162686 6695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:40:20.162830 6688 server_base.cc:1034] running on GCE node
I20260806 11:40:20.163566 6688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:40:20.164927 6688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:40:20.166221 6688 hybrid_clock.cc:648] HybridClock initialized: now 1786016420166196 us; error 40 us; skew 500 ppm
Aug 06 11:40:20 dist-test-slave-7c35 krb5kdc[5543](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016420, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:40:20.169842 6688 init.cc:377] Logged in from keytab as kudu/127.21.14.254@KRBTEST.COM (short username kudu)
I20260806 11:40:20.171149 6688 webserver.cc:460] Webserver started at http://127.21.14.254:36065/ using document root <none> and password file <none>
I20260806 11:40:20.171820 6688 fs_manager.cc:359] Metadata directory not provided
I20260806 11:40:20.171929 6688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:40:20.176615 6688 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 11:40:20.179592 6704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:20.181509 6688 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 11:40:20.181751 6688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/master-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/master-0/wal
uuid: "62e3c47d9e32427eb798272b95ca13a7"
format_stamp: "Formatted at 2026-08-06 11:39:37 on dist-test-slave-7c35"
server_key: "1e45a4d2d9bff82ef252e9bb15b9e105"
server_key_iv: "04cfce1125d7b46e3c555caf4d012c86"
server_key_version: "encryptionkey@0"
I20260806 11:40:20.182430 6688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:40:20.196843 6688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:40:20.200884 6688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:40:20.201308 6688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:40:20.210151 6688 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.254:36681
I20260806 11:40:20.210456 6762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.254:36681 every 8 connection(s)
I20260806 11:40:20.211549 6688 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
I20260806 11:40:20.216516 6763 sys_catalog.cc:263] Verifying existing consensus state
I20260806 11:40:20.218423 6763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62e3c47d9e32427eb798272b95ca13a7: Bootstrap starting.
I20260806 11:40:20.221872 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 6688
I20260806 11:40:20.222425 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.193:39037
--local_ip_for_outbound_sockets=127.21.14.193
--tserver_master_addrs=127.21.14.254:36681
--webserver_port=38685
--webserver_interface=127.21.14.193
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab
--principal=kudu/127.21.14.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:43983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 11:40:20.239262 6763 log.cc:824] T 00000000000000000000000000000000 P 62e3c47d9e32427eb798272b95ca13a7: Log is configured to *not* fsync() on all Append() calls
I20260806 11:40:20.251716 6763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62e3c47d9e32427eb798272b95ca13a7: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 11:40:20.252187 6763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62e3c47d9e32427eb798272b95ca13a7: Bootstrap complete.
I20260806 11:40:20.256992 6763 raft_consensus.cc:348] T 00000000000000000000000000000000 P 62e3c47d9e32427eb798272b95ca13a7 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62e3c47d9e32427eb798272b95ca13a7" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 36681 } }
I20260806 11:40:20.257464 6763 raft_consensus.cc:729] T 00000000000000000000000000000000 P 62e3c47d9e32427eb798272b95ca13a7 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62e3c47d9e32427eb798272b95ca13a7, State: Initialized, Role: FOLLOWER
I20260806 11:40:20.258111 6763 consensus_queue.cc:260] T 00000000000000000000000000000000 P 62e3c47d9e32427eb798272b95ca13a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62e3c47d9e32427eb798272b95ca13a7" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 36681 } }
I20260806 11:40:20.258275 6763 raft_consensus.cc:388] T 00000000000000000000000000000000 P 62e3c47d9e32427eb798272b95ca13a7 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 11:40:20.258374 6763 raft_consensus.cc:482] T 00000000000000000000000000000000 P 62e3c47d9e32427eb798272b95ca13a7 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 11:40:20.258494 6763 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 62e3c47d9e32427eb798272b95ca13a7 [term 2 FOLLOWER]: Advancing to term 3
I20260806 11:40:20.260648 6763 raft_consensus.cc:504] T 00000000000000000000000000000000 P 62e3c47d9e32427eb798272b95ca13a7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62e3c47d9e32427eb798272b95ca13a7" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 36681 } }
I20260806 11:40:20.261032 6763 leader_election.cc:302] T 00000000000000000000000000000000 P 62e3c47d9e32427eb798272b95ca13a7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 62e3c47d9e32427eb798272b95ca13a7; no voters:
I20260806 11:40:20.261449 6763 leader_election.cc:288] T 00000000000000000000000000000000 P 62e3c47d9e32427eb798272b95ca13a7 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 11:40:20.261544 6767 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 62e3c47d9e32427eb798272b95ca13a7 [term 3 FOLLOWER]: Leader election won for term 3
I20260806 11:40:20.261765 6767 raft_consensus.cc:686] T 00000000000000000000000000000000 P 62e3c47d9e32427eb798272b95ca13a7 [term 3 LEADER]: Becoming Leader. State: Replica: 62e3c47d9e32427eb798272b95ca13a7, State: Running, Role: LEADER
I20260806 11:40:20.262086 6767 consensus_queue.cc:237] T 00000000000000000000000000000000 P 62e3c47d9e32427eb798272b95ca13a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62e3c47d9e32427eb798272b95ca13a7" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 36681 } }
I20260806 11:40:20.262797 6763 sys_catalog.cc:564] T 00000000000000000000000000000000 P 62e3c47d9e32427eb798272b95ca13a7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 11:40:20.263550 6768 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62e3c47d9e32427eb798272b95ca13a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "62e3c47d9e32427eb798272b95ca13a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62e3c47d9e32427eb798272b95ca13a7" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 36681 } } }
I20260806 11:40:20.263571 6769 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62e3c47d9e32427eb798272b95ca13a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 62e3c47d9e32427eb798272b95ca13a7. Latest consensus state: current_term: 3 leader_uuid: "62e3c47d9e32427eb798272b95ca13a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62e3c47d9e32427eb798272b95ca13a7" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 36681 } } }
I20260806 11:40:20.263705 6768 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62e3c47d9e32427eb798272b95ca13a7 [sys.catalog]: This master's current role is: LEADER
I20260806 11:40:20.263707 6769 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62e3c47d9e32427eb798272b95ca13a7 [sys.catalog]: This master's current role is: LEADER
I20260806 11:40:20.264075 6771 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 11:40:20.269657 6771 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 11:40:20.270550 6771 catalog_manager.cc:1259] Loaded cluster ID: 1f5eded52c3a49deab2b799491110e1f
I20260806 11:40:20.270617 6771 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 11:40:20.273337 6771 catalog_manager.cc:1504] Loading token signing keys...
I20260806 11:40:20.274225 6771 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 62e3c47d9e32427eb798272b95ca13a7: Loaded TSK: 0
I20260806 11:40:20.274327 6771 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 62e3c47d9e32427eb798272b95ca13a7: Loaded TSK: 1
F20260806 11:40:20.275308 6771 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f9dadfc29ad google::LogMessage::Fail() at ??:0
@ 0x7f9dadfc588a google::LogMessage::SendToLog() at ??:0
@ 0x7f9dadfc24de google::LogMessage::Flush() at ??:0
@ 0x7f9dadfc3a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f9db0c8618d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f9db0c7ef0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f9db89277bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f9db890199c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f9db8902178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7f9db8943df8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f9db896282b std::function<>::operator()() at ??:0
@ 0x7f9db8901b4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f9db8902f3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f9db88ff579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f9db8943640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f9db02143d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f9db0214c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f9db02163f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f9db0205512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f9db18496db start_thread at ??:0
@ 0x7f9dacac061f clone at ??:0
W20260806 11:40:20.392233 6765 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:40:20.392545 6765 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:40:20.392592 6765 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:40:20.396981 6765 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:40:20.397065 6765 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:40:20.397136 6765 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.193
I20260806 11:40:20.403115 6765 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:43983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab
--principal=kudu/127.21.14.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.193:39037
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.14.193
--webserver_port=38685
--tserver_master_addrs=127.21.14.254:36681
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6765
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.193
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:40:20.404369 6765 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:40:20.405490 6765 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:40:20.416323 6815 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:40:20.416483 6765 server_base.cc:1034] running on GCE node
W20260806 11:40:20.416347 6817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:40:20.416329 6814 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:40:20.417212 6765 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:40:20.417937 6765 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:40:20.419139 6765 hybrid_clock.cc:648] HybridClock initialized: now 1786016420419110 us; error 48 us; skew 500 ppm
Aug 06 11:40:20 dist-test-slave-7c35 krb5kdc[5543](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016420, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:40:20.422374 6765 init.cc:377] Logged in from keytab as kudu/127.21.14.193@KRBTEST.COM (short username kudu)
I20260806 11:40:20.423552 6765 webserver.cc:460] Webserver started at http://127.21.14.193:38685/ using document root <none> and password file <none>
I20260806 11:40:20.424207 6765 fs_manager.cc:359] Metadata directory not provided
I20260806 11:40:20.424317 6765 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:40:20.429593 6765 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 11:40:20.433321 6824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:20.435319 6765 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 11:40:20.435516 6765 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-0/wal
uuid: "a44ebc4b518a42d5a327cfda159f9635"
format_stamp: "Formatted at 2026-08-06 11:39:37 on dist-test-slave-7c35"
server_key: "f44a7c8cf2c40157cdef5ab6e9d9e01f"
server_key_iv: "400c053b1e170a40f3b2fecf3722e5e7"
server_key_version: "encryptionkey@0"
I20260806 11:40:20.436187 6765 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:40:20.456100 6765 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:40:20.459914 6765 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:40:20.460165 6765 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:40:20.460860 6765 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:40:20.462208 6832 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 11:40:20.468329 6765 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 11:40:20.468465 6765 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.007s user 0.001s sys 0.000s
I20260806 11:40:20.468622 6765 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 11:40:20.470628 6765 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 11:40:20.470726 6765 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260806 11:40:20.471117 6832 tablet_bootstrap.cc:492] T 69fef14c78034517899675c07573120e P a44ebc4b518a42d5a327cfda159f9635: Bootstrap starting.
I20260806 11:40:20.482249 6765 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.193:39037
I20260806 11:40:20.482647 6947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.193:39037 every 8 connection(s)
I20260806 11:40:20.483511 6765 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
I20260806 11:40:20.492522 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 6765
I20260806 11:40:20.493049 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.194:43875
--local_ip_for_outbound_sockets=127.21.14.194
--tserver_master_addrs=127.21.14.254:36681
--webserver_port=36047
--webserver_interface=127.21.14.194
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab
--principal=kudu/127.21.14.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:43983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 11:40:20.495107 6832 log.cc:824] T 69fef14c78034517899675c07573120e P a44ebc4b518a42d5a327cfda159f9635: Log is configured to *not* fsync() on all Append() calls
I20260806 11:40:20.503304 6832 tablet_bootstrap.cc:492] T 69fef14c78034517899675c07573120e P a44ebc4b518a42d5a327cfda159f9635: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 11:40:20.503760 6832 tablet_bootstrap.cc:492] T 69fef14c78034517899675c07573120e P a44ebc4b518a42d5a327cfda159f9635: Bootstrap complete.
I20260806 11:40:20.505010 6832 ts_tablet_manager.cc:1397] T 69fef14c78034517899675c07573120e P a44ebc4b518a42d5a327cfda159f9635: Time spent bootstrapping tablet: real 0.034s user 0.025s sys 0.007s
I20260806 11:40:20.507697 6832 raft_consensus.cc:348] T 69fef14c78034517899675c07573120e P a44ebc4b518a42d5a327cfda159f9635 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558d5fa7ae6d48058c7e74f21bc12211" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 43875 } } peers { permanent_uuid: "9678ece9548c429ba9076df50c10b444" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 36077 } } peers { permanent_uuid: "a44ebc4b518a42d5a327cfda159f9635" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 39037 } }
I20260806 11:40:20.508188 6832 raft_consensus.cc:729] T 69fef14c78034517899675c07573120e P a44ebc4b518a42d5a327cfda159f9635 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a44ebc4b518a42d5a327cfda159f9635, State: Initialized, Role: FOLLOWER
I20260806 11:40:20.508737 6832 consensus_queue.cc:260] T 69fef14c78034517899675c07573120e P a44ebc4b518a42d5a327cfda159f9635 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558d5fa7ae6d48058c7e74f21bc12211" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 43875 } } peers { permanent_uuid: "9678ece9548c429ba9076df50c10b444" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 36077 } } peers { permanent_uuid: "a44ebc4b518a42d5a327cfda159f9635" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 39037 } }
I20260806 11:40:20.509840 6832 ts_tablet_manager.cc:1428] T 69fef14c78034517899675c07573120e P a44ebc4b518a42d5a327cfda159f9635: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
W20260806 11:40:20.535600 6950 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:40:20.485741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:36681 (local address 127.21.14.193:54709)
0806 11:40:20.486301 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:20.486359 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 11:40:20.487342 (+ 983us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:40:20.534541 (+ 47199us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:36681: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":182,"threads_started":1}
W20260806 11:40:20.536290 6948 heartbeater.cc:644] Failed to heartbeat to 127.21.14.254:36681 (0 consecutive failures): Network error: Failed to ping master at 127.21.14.254:36681: Client connection negotiation failed: client connection to 127.21.14.254:36681: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 11:40:20.537142 6950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:20.536816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:36681 (local address 127.21.14.193:51983)
0806 11:40:20.536987 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:20.537069 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:36681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260806 11:40:20.537921 6950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:20.537666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:36681 (local address 127.21.14.193:51627)
0806 11:40:20.537796 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:20.537850 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:36681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260806 11:40:20.538130 6948 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 11:40:20.631474 6951 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:40:20.631866 6951 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:40:20.631958 6951 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:40:20.636446 6951 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:40:20.636560 6951 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:40:20.636665 6951 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.194
I20260806 11:40:20.642792 6951 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:43983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab
--principal=kudu/127.21.14.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.194:43875
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.14.194
--webserver_port=36047
--tserver_master_addrs=127.21.14.254:36681
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6951
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.194
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:40:20.644186 6951 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:40:20.645357 6951 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:40:20.656436 6959 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:40:20.656428 6958 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:40:20.656463 6961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:40:20.656662 6951 server_base.cc:1034] running on GCE node
I20260806 11:40:20.657375 6951 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:40:20.658238 6951 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:40:20.659487 6951 hybrid_clock.cc:648] HybridClock initialized: now 1786016420659446 us; error 54 us; skew 500 ppm
Aug 06 11:40:20 dist-test-slave-7c35 krb5kdc[5543](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016420, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:40:20.662647 6951 init.cc:377] Logged in from keytab as kudu/127.21.14.194@KRBTEST.COM (short username kudu)
I20260806 11:40:20.663784 6951 webserver.cc:460] Webserver started at http://127.21.14.194:36047/ using document root <none> and password file <none>
I20260806 11:40:20.664422 6951 fs_manager.cc:359] Metadata directory not provided
I20260806 11:40:20.664526 6951 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:40:20.668913 6951 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.002s
I20260806 11:40:20.671317 6968 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:20.672689 6951 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 11:40:20.672854 6951 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-1/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-1/wal
uuid: "558d5fa7ae6d48058c7e74f21bc12211"
format_stamp: "Formatted at 2026-08-06 11:39:37 on dist-test-slave-7c35"
server_key: "dd063a7e8a5d6511a2de98e97fa932ec"
server_key_iv: "deb9c4d2f30c84395fdf40128c335312"
server_key_version: "encryptionkey@0"
I20260806 11:40:20.673473 6951 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:40:20.715415 6951 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:40:20.720105 6951 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:40:20.720363 6951 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:40:20.721045 6951 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:40:20.721987 6951 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:40:20.722078 6951 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:20.722179 6951 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:40:20.722234 6951 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:20.732228 6951 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.194:43875
I20260806 11:40:20.732262 7087 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.194:43875 every 8 connection(s)
I20260806 11:40:20.733326 6951 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
I20260806 11:40:20.733695 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 6951
I20260806 11:40:20.734241 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.195:36077
--local_ip_for_outbound_sockets=127.21.14.195
--tserver_master_addrs=127.21.14.254:36681
--webserver_port=32799
--webserver_interface=127.21.14.195
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab
--principal=kudu/127.21.14.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:43983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 11:40:20.738487 7091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:20.736018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:36681 (local address 127.21.14.194:54241)
0806 11:40:20.736687 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:20.736855 (+ 168us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:36681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":153,"threads_started":1}
W20260806 11:40:20.739058 7088 heartbeater.cc:644] Failed to heartbeat to 127.21.14.254:36681 (0 consecutive failures): Network error: Failed to ping master at 127.21.14.254:36681: Client connection negotiation failed: client connection to 127.21.14.254:36681: connect: Connection refused (error 111)
I20260806 11:40:20.739917 7091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:20.739611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:36681 (local address 127.21.14.194:46447)
0806 11:40:20.739760 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:20.739816 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:36681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260806 11:40:20.740700 7091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:20.740462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:36681 (local address 127.21.14.194:43425)
0806 11:40:20.740593 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:20.740646 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:36681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260806 11:40:20.741098 7088 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 11:40:20.861481 7090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:40:20.861845 7090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:40:20.861896 7090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:40:20.866312 7090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:40:20.866400 7090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:40:20.866465 7090 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.195
I20260806 11:40:20.872298 7090 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:43983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab
--principal=kudu/127.21.14.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.195:36077
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.14.195
--webserver_port=32799
--tserver_master_addrs=127.21.14.254:36681
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7090
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.195
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:40:20.873584 7090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:40:20.874740 7090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:40:20.886072 7100 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:40:20.886020 7097 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:40:20.886071 7098 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:40:20.886041 7090 server_base.cc:1034] running on GCE node
I20260806 11:40:20.886829 7090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:40:20.888023 7090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:40:20.889236 7090 hybrid_clock.cc:648] HybridClock initialized: now 1786016420889198 us; error 54 us; skew 500 ppm
Aug 06 11:40:20 dist-test-slave-7c35 krb5kdc[5543](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016420, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:40:20.892517 7090 init.cc:377] Logged in from keytab as kudu/127.21.14.195@KRBTEST.COM (short username kudu)
I20260806 11:40:20.893616 7090 webserver.cc:460] Webserver started at http://127.21.14.195:32799/ using document root <none> and password file <none>
I20260806 11:40:20.894315 7090 fs_manager.cc:359] Metadata directory not provided
I20260806 11:40:20.894430 7090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:40:20.899616 7090 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 11:40:20.901966 7107 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:20.903420 7090 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 11:40:20.903579 7090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-2/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-2/wal
uuid: "9678ece9548c429ba9076df50c10b444"
format_stamp: "Formatted at 2026-08-06 11:39:37 on dist-test-slave-7c35"
server_key: "6b73a94d75cc4e1fa264b6b69f5d24fe"
server_key_iv: "494109ca392fea40387785f02eebeb71"
server_key_version: "encryptionkey@0"
I20260806 11:40:20.904188 7090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:40:20.926414 7090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:40:20.929736 7090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:40:20.929972 7090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:40:20.930668 7090 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:40:20.931644 7090 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:40:20.931725 7090 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:20.931800 7090 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:40:20.931846 7090 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:20.941395 7090 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.195:36077
I20260806 11:40:20.941423 7224 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.195:36077 every 8 connection(s)
I20260806 11:40:20.942474 7090 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
I20260806 11:40:20.944289 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 7090
I20260806 11:40:20.946794 7230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:20.944866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:36681 (local address 127.21.14.195:57021)
0806 11:40:20.945446 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:20.945591 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:36681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":144,"threads_started":1}
W20260806 11:40:20.947428 7225 heartbeater.cc:644] Failed to heartbeat to 127.21.14.254:36681 (0 consecutive failures): Network error: Failed to ping master at 127.21.14.254:36681: Client connection negotiation failed: client connection to 127.21.14.254:36681: connect: Connection refused (error 111)
I20260806 11:40:20.948061 7230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:20.947809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:36681 (local address 127.21.14.195:58777)
0806 11:40:20.947939 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:20.947996 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:36681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260806 11:40:20.948772 7230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:20.948559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:36681 (local address 127.21.14.195:50641)
0806 11:40:20.948669 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:20.948717 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:36681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260806 11:40:20.949050 7225 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 11:40:21.539419 7231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:21.538818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:36681 (local address 127.21.14.193:43781)
0806 11:40:21.539220 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:21.539291 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:36681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":127,"threads_started":1}
I20260806 11:40:21.742306 7232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:21.741811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:36681 (local address 127.21.14.194:59219)
0806 11:40:21.742111 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:21.742177 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:36681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":87,"threads_started":1}
I20260806 11:40:21.950387 7233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:21.949818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:36681 (local address 127.21.14.195:51673)
0806 11:40:21.950194 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:21.950262 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:36681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":100,"threads_started":1}
I20260806 11:40:22.039084 7234 raft_consensus.cc:482] T 69fef14c78034517899675c07573120e P a44ebc4b518a42d5a327cfda159f9635 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 11:40:22.039458 7234 raft_consensus.cc:504] T 69fef14c78034517899675c07573120e P a44ebc4b518a42d5a327cfda159f9635 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558d5fa7ae6d48058c7e74f21bc12211" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 43875 } } peers { permanent_uuid: "9678ece9548c429ba9076df50c10b444" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 36077 } } peers { permanent_uuid: "a44ebc4b518a42d5a327cfda159f9635" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 39037 } }
I20260806 11:40:22.040827 7234 leader_election.cc:288] T 69fef14c78034517899675c07573120e P a44ebc4b518a42d5a327cfda159f9635 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 558d5fa7ae6d48058c7e74f21bc12211 (127.21.14.194:43875), 9678ece9548c429ba9076df50c10b444 (127.21.14.195:36077)
Aug 06 11:40:22 dist-test-slave-7c35 krb5kdc[5543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016420, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.193@KRBTEST.COM for kudu/127.21.14.194@KRBTEST.COM
Aug 06 11:40:22 dist-test-slave-7c35 krb5kdc[5543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016420, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.193@KRBTEST.COM for kudu/127.21.14.195@KRBTEST.COM
I20260806 11:40:22.058399 7236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:22.041399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:51767 (local address 127.21.14.194:43875)
0806 11:40:22.041901 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:22.041931 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:22.041989 (+ 58us) server_negotiation.cc:407] Connection header received
0806 11:40:22.042728 (+ 739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:22.042734 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:22.043183 (+ 449us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:22.043508 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:40:22.044900 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:22.045892 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:22.047242 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:22.047728 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:22.047925 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:22.050980 (+ 3055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:40:22.050986 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:40:22.051003 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:40:22.051037 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:40:22.053830 (+ 2793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:40:22.054938 (+ 1108us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:40:22.054945 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:40:22.054952 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:40:22.055203 (+ 251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:40:22.055727 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:40:22.055731 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:40:22.055733 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:40:22.056481 (+ 748us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:40:22.056631 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:22.057686 (+ 1055us) server_negotiation.cc:299] Negotiation successful
0806 11:40:22.058004 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":130,"threads_started":1}
I20260806 11:40:22.058535 7237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:22.041480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.195:36077 (local address 127.21.14.193:58207)
0806 11:40:22.041893 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:22.041912 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 11:40:22.042037 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:40:22.043571 (+ 1534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:40:22.043577 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:40:22.044160 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:40:22.044690 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:22.044708 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:22.046069 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:22.046072 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:22.047086 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:22.047096 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:22.047873 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:22.047877 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:22.048118 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:22.048904 (+ 786us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:40:22.048917 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:40:22.051030 (+ 2113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:40:22.054026 (+ 2996us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:40:22.054035 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:40:22.054052 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:40:22.054737 (+ 685us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:40:22.055333 (+ 596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:40:22.055337 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:40:22.055340 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:40:22.055602 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:40:22.056695 (+ 1093us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:40:22.056702 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:40:22.057096 (+ 394us) client_negotiation.cc:769] Sending connection context
0806 11:40:22.057586 (+ 490us) client_negotiation.cc:240] Negotiation successful
0806 11:40:22.058033 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"mutex_wait_us":370,"spinlock_wait_cycles":52864,"thread_start_us":53,"threads_started":1}
I20260806 11:40:22.058370 7238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:22.041644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:58207 (local address 127.21.14.195:36077)
0806 11:40:22.042009 (+ 365us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:22.042032 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:22.042076 (+ 44us) server_negotiation.cc:407] Connection header received
0806 11:40:22.042770 (+ 694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:22.042776 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:22.043183 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:22.043514 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:40:22.044888 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:22.045885 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:22.047218 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:22.047729 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:22.047924 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:22.051254 (+ 3330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:40:22.051261 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:40:22.051276 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:40:22.051315 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:40:22.053888 (+ 2573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:40:22.054938 (+ 1050us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:40:22.054945 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:40:22.054952 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:40:22.055194 (+ 242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:40:22.055727 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:40:22.055731 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:40:22.055733 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:40:22.056481 (+ 748us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:40:22.056631 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:22.057650 (+ 1019us) server_negotiation.cc:299] Negotiation successful
0806 11:40:22.057992 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":116,"threads_started":1}
I20260806 11:40:22.059418 7235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:22.041045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.194:43875 (local address 127.21.14.193:51767)
0806 11:40:22.041444 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:22.041460 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 11:40:22.041602 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:40:22.043571 (+ 1969us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:40:22.043577 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:40:22.044148 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:40:22.044689 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:22.044707 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:22.046082 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:22.046085 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:22.047100 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:22.047111 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:22.047873 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:22.047877 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:22.048120 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:22.048859 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:40:22.048877 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:40:22.050610 (+ 1733us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:40:22.054026 (+ 3416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:40:22.054035 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:40:22.054052 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:40:22.054737 (+ 685us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:40:22.055333 (+ 596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:40:22.055337 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:40:22.055340 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:40:22.055602 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:40:22.056703 (+ 1101us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:40:22.056706 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:40:22.057096 (+ 390us) client_negotiation.cc:769] Sending connection context
0806 11:40:22.057467 (+ 371us) client_negotiation.cc:240] Negotiation successful
0806 11:40:22.059248 (+ 1781us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"mutex_wait_us":248,"spinlock_wait_cycles":44928,"thread_start_us":69,"threads_started":1}
W20260806 11:40:22.061733 6828 leader_election.cc:341] T 69fef14c78034517899675c07573120e P a44ebc4b518a42d5a327cfda159f9635 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9678ece9548c429ba9076df50c10b444 (127.21.14.195:36077): Not found: Tablet not found: 69fef14c78034517899675c07573120e
W20260806 11:40:22.061955 6825 leader_election.cc:341] T 69fef14c78034517899675c07573120e P a44ebc4b518a42d5a327cfda159f9635 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 558d5fa7ae6d48058c7e74f21bc12211 (127.21.14.194:43875): Not found: Tablet not found: 69fef14c78034517899675c07573120e
I20260806 11:40:22.062060 6825 leader_election.cc:302] T 69fef14c78034517899675c07573120e P a44ebc4b518a42d5a327cfda159f9635 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a44ebc4b518a42d5a327cfda159f9635; no voters: 558d5fa7ae6d48058c7e74f21bc12211, 9678ece9548c429ba9076df50c10b444
I20260806 11:40:22.062372 7234 raft_consensus.cc:2726] T 69fef14c78034517899675c07573120e P a44ebc4b518a42d5a327cfda159f9635 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 11:40:22.540793 7235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:22.540455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:36681 (local address 127.21.14.193:36551)
0806 11:40:22.540626 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:22.540696 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:36681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260806 11:40:22.744002 7239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:22.743431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:36681 (local address 127.21.14.194:48943)
0806 11:40:22.743789 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:22.743864 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:36681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":93,"threads_started":1}
I20260806 11:40:22.951761 7240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:22.951324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:36681 (local address 127.21.14.195:56719)
0806 11:40:22.951599 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:22.951657 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:36681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":89,"threads_started":1}
I20260806 11:40:23.542351 7241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:23.541846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:36681 (local address 127.21.14.193:34387)
0806 11:40:23.542162 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:23.542231 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:36681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":102,"threads_started":1}
I20260806 11:40:23.745682 7242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:23.745039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:36681 (local address 127.21.14.194:58665)
0806 11:40:23.745429 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:23.745547 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:36681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":86,"threads_started":1}
I20260806 11:40:23.953200 7243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:23.952702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:36681 (local address 127.21.14.195:60427)
0806 11:40:23.953017 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:23.953087 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:36681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":90,"threads_started":1}
I20260806 11:40:24.144960 7244 raft_consensus.cc:482] T 69fef14c78034517899675c07573120e P a44ebc4b518a42d5a327cfda159f9635 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 11:40:24.145113 7244 raft_consensus.cc:504] T 69fef14c78034517899675c07573120e P a44ebc4b518a42d5a327cfda159f9635 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558d5fa7ae6d48058c7e74f21bc12211" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 43875 } } peers { permanent_uuid: "9678ece9548c429ba9076df50c10b444" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 36077 } } peers { permanent_uuid: "a44ebc4b518a42d5a327cfda159f9635" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 39037 } }
I20260806 11:40:24.145742 7244 leader_election.cc:288] T 69fef14c78034517899675c07573120e P a44ebc4b518a42d5a327cfda159f9635 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 558d5fa7ae6d48058c7e74f21bc12211 (127.21.14.194:43875), 9678ece9548c429ba9076df50c10b444 (127.21.14.195:36077)
W20260806 11:40:24.146780 6825 leader_election.cc:341] T 69fef14c78034517899675c07573120e P a44ebc4b518a42d5a327cfda159f9635 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 558d5fa7ae6d48058c7e74f21bc12211 (127.21.14.194:43875): Not found: Tablet not found: 69fef14c78034517899675c07573120e
W20260806 11:40:24.147019 6828 leader_election.cc:341] T 69fef14c78034517899675c07573120e P a44ebc4b518a42d5a327cfda159f9635 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9678ece9548c429ba9076df50c10b444 (127.21.14.195:36077): Not found: Tablet not found: 69fef14c78034517899675c07573120e
I20260806 11:40:24.147123 6828 leader_election.cc:302] T 69fef14c78034517899675c07573120e P a44ebc4b518a42d5a327cfda159f9635 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a44ebc4b518a42d5a327cfda159f9635; no voters: 558d5fa7ae6d48058c7e74f21bc12211, 9678ece9548c429ba9076df50c10b444
I20260806 11:40:24.147330 7244 raft_consensus.cc:2726] T 69fef14c78034517899675c07573120e P a44ebc4b518a42d5a327cfda159f9635 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 11:40:24.543911 7245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:24.543386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:36681 (local address 127.21.14.193:35229)
0806 11:40:24.543692 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:24.543785 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:36681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":119,"threads_started":1}
I20260806 11:40:24.747061 7246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:24.746520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:36681 (local address 127.21.14.194:57221)
0806 11:40:24.746811 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:24.746952 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:36681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":79,"threads_started":1}
I20260806 11:40:24.954957 7247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:24.954370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:36681 (local address 127.21.14.195:50001)
0806 11:40:24.954680 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:24.954748 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:36681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":85,"threads_started":1}
I20260806 11:40:25.545636 7248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:25.545095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:36681 (local address 127.21.14.193:59763)
0806 11:40:25.545464 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:25.545530 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:36681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":128,"threads_started":1}
I20260806 11:40:25.748477 7249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:25.747976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:36681 (local address 127.21.14.194:36397)
0806 11:40:25.748297 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:25.748369 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:36681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":89,"threads_started":1}
I20260806 11:40:25.759696 7250 raft_consensus.cc:482] T 69fef14c78034517899675c07573120e P a44ebc4b518a42d5a327cfda159f9635 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 11:40:25.759883 7250 raft_consensus.cc:504] T 69fef14c78034517899675c07573120e P a44ebc4b518a42d5a327cfda159f9635 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558d5fa7ae6d48058c7e74f21bc12211" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 43875 } } peers { permanent_uuid: "9678ece9548c429ba9076df50c10b444" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 36077 } } peers { permanent_uuid: "a44ebc4b518a42d5a327cfda159f9635" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 39037 } }
I20260806 11:40:25.760442 7250 leader_election.cc:288] T 69fef14c78034517899675c07573120e P a44ebc4b518a42d5a327cfda159f9635 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 558d5fa7ae6d48058c7e74f21bc12211 (127.21.14.194:43875), 9678ece9548c429ba9076df50c10b444 (127.21.14.195:36077)
W20260806 11:40:25.761426 6825 leader_election.cc:341] T 69fef14c78034517899675c07573120e P a44ebc4b518a42d5a327cfda159f9635 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 558d5fa7ae6d48058c7e74f21bc12211 (127.21.14.194:43875): Not found: Tablet not found: 69fef14c78034517899675c07573120e
W20260806 11:40:25.761533 6828 leader_election.cc:341] T 69fef14c78034517899675c07573120e P a44ebc4b518a42d5a327cfda159f9635 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9678ece9548c429ba9076df50c10b444 (127.21.14.195:36077): Not found: Tablet not found: 69fef14c78034517899675c07573120e
I20260806 11:40:25.761672 6828 leader_election.cc:302] T 69fef14c78034517899675c07573120e P a44ebc4b518a42d5a327cfda159f9635 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a44ebc4b518a42d5a327cfda159f9635; no voters: 558d5fa7ae6d48058c7e74f21bc12211, 9678ece9548c429ba9076df50c10b444
I20260806 11:40:25.761862 7250 raft_consensus.cc:2726] T 69fef14c78034517899675c07573120e P a44ebc4b518a42d5a327cfda159f9635 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 11:40:25.956355 7251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:25.955874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:36681 (local address 127.21.14.195:57807)
0806 11:40:25.956183 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:25.956247 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:36681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":86,"threads_started":1}
I20260806 11:40:26.547322 7252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:26.546723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:36681 (local address 127.21.14.193:55275)
0806 11:40:26.547069 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:26.547184 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:36681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":158,"threads_started":1}
I20260806 11:40:26.750141 7253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:26.749559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:36681 (local address 127.21.14.194:50133)
0806 11:40:26.749941 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:26.750011 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:36681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":136,"threads_started":1}
I20260806 11:40:26.957844 7254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:26.957322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:36681 (local address 127.21.14.195:55425)
0806 11:40:26.957662 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:26.957737 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:36681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":87,"threads_started":1}
I20260806 11:40:27.548884 7255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:27.548330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:36681 (local address 127.21.14.193:43741)
0806 11:40:27.548690 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:27.548763 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:36681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":105,"threads_started":1}
I20260806 11:40:27.751628 7256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:27.751112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:36681 (local address 127.21.14.194:36843)
0806 11:40:27.751458 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:27.751522 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:36681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":82,"threads_started":1}
I20260806 11:40:27.959558 7257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:27.958962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:36681 (local address 127.21.14.195:35281)
0806 11:40:27.959344 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:27.959416 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:36681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":92,"threads_started":1}
I20260806 11:40:28.550419 7258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:28.549925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:36681 (local address 127.21.14.193:45271)
0806 11:40:28.550238 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:28.550309 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:36681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":93,"threads_started":1}
I20260806 11:40:28.753163 7259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:28.752675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:36681 (local address 127.21.14.194:55787)
0806 11:40:28.752989 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:28.753057 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:36681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":107,"threads_started":1}
I20260806 11:40:28.960945 7260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:28.960495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:36681 (local address 127.21.14.195:35143)
0806 11:40:28.960781 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:28.960840 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:36681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":77,"threads_started":1}
I20260806 11:40:29.551821 7261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:29.551383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:36681 (local address 127.21.14.193:59009)
0806 11:40:29.551653 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:29.551720 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:36681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":83,"threads_started":1}
I20260806 11:40:29.754454 7262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:29.753941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:36681 (local address 127.21.14.194:54045)
0806 11:40:29.754277 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:29.754341 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:36681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":88,"threads_started":1}
I20260806 11:40:29.925714 7263 raft_consensus.cc:482] T 69fef14c78034517899675c07573120e P a44ebc4b518a42d5a327cfda159f9635 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 11:40:29.925957 7263 raft_consensus.cc:504] T 69fef14c78034517899675c07573120e P a44ebc4b518a42d5a327cfda159f9635 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558d5fa7ae6d48058c7e74f21bc12211" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 43875 } } peers { permanent_uuid: "9678ece9548c429ba9076df50c10b444" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 36077 } } peers { permanent_uuid: "a44ebc4b518a42d5a327cfda159f9635" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 39037 } }
I20260806 11:40:29.926391 7263 leader_election.cc:288] T 69fef14c78034517899675c07573120e P a44ebc4b518a42d5a327cfda159f9635 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 558d5fa7ae6d48058c7e74f21bc12211 (127.21.14.194:43875), 9678ece9548c429ba9076df50c10b444 (127.21.14.195:36077)
W20260806 11:40:29.927305 6825 leader_election.cc:341] T 69fef14c78034517899675c07573120e P a44ebc4b518a42d5a327cfda159f9635 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 558d5fa7ae6d48058c7e74f21bc12211 (127.21.14.194:43875): Not found: Tablet not found: 69fef14c78034517899675c07573120e
W20260806 11:40:29.927407 6828 leader_election.cc:341] T 69fef14c78034517899675c07573120e P a44ebc4b518a42d5a327cfda159f9635 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9678ece9548c429ba9076df50c10b444 (127.21.14.195:36077): Not found: Tablet not found: 69fef14c78034517899675c07573120e
I20260806 11:40:29.927505 6828 leader_election.cc:302] T 69fef14c78034517899675c07573120e P a44ebc4b518a42d5a327cfda159f9635 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a44ebc4b518a42d5a327cfda159f9635; no voters: 558d5fa7ae6d48058c7e74f21bc12211, 9678ece9548c429ba9076df50c10b444
I20260806 11:40:29.927716 7263 raft_consensus.cc:2726] T 69fef14c78034517899675c07573120e P a44ebc4b518a42d5a327cfda159f9635 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 11:40:29.963003 7264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:29.962125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:36681 (local address 127.21.14.195:39577)
0806 11:40:29.962700 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:29.962782 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:36681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":116,"threads_started":1}
I20260806 11:40:30.553483 7265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:30.552945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:36681 (local address 127.21.14.193:33725)
0806 11:40:30.553304 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:30.553375 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:36681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":92,"threads_started":1}
I20260806 11:40:30.756197 7266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:30.755698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:36681 (local address 127.21.14.194:51501)
0806 11:40:30.756009 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:30.756091 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:36681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":82,"threads_started":1}
I20260806 11:40:30.964576 7267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:30.963998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:36681 (local address 127.21.14.195:57499)
0806 11:40:30.964395 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:30.964465 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:36681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":110,"threads_started":1}
I20260806 11:40:31.554962 7268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:31.554475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:36681 (local address 127.21.14.193:51121)
0806 11:40:31.554764 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:31.554828 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:36681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":84,"threads_started":1}
I20260806 11:40:31.757666 7270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:31.757150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:36681 (local address 127.21.14.194:54149)
0806 11:40:31.757467 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:31.757561 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:36681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":89,"threads_started":1}
I20260806 11:40:31.966012 7271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:31.965543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:36681 (local address 127.21.14.195:34987)
0806 11:40:31.965819 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:31.965878 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:36681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":82,"threads_started":1}
I20260806 11:40:32.556881 7272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:32.556189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:36681 (local address 127.21.14.193:35049)
0806 11:40:32.556633 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:32.556704 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:36681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":101,"threads_started":1}
I20260806 11:40:32.759249 7273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:32.758726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:36681 (local address 127.21.14.194:45653)
0806 11:40:32.759066 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:32.759137 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:36681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":170,"threads_started":1}
I20260806 11:40:32.967634 7274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:32.967124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:36681 (local address 127.21.14.195:56609)
0806 11:40:32.967460 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:32.967527 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:36681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":101,"threads_started":1}
I20260806 11:40:33.558478 7275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:33.557992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:36681 (local address 127.21.14.193:55587)
0806 11:40:33.558303 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:33.558371 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:36681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":98,"threads_started":1}
I20260806 11:40:33.760843 7276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:33.760336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:36681 (local address 127.21.14.194:60029)
0806 11:40:33.760670 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:33.760737 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:36681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":100,"threads_started":1}
I20260806 11:40:33.969234 7277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:33.968672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:36681 (local address 127.21.14.195:52791)
0806 11:40:33.968995 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:33.969104 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:36681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":91,"threads_started":1}
I20260806 11:40:34.559890 7278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:34.559394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:36681 (local address 127.21.14.193:44179)
0806 11:40:34.559716 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:34.559784 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:36681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":86,"threads_started":1}
I20260806 11:40:34.762331 7279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:34.761815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:36681 (local address 127.21.14.194:43807)
0806 11:40:34.762153 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:34.762224 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:36681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":99,"threads_started":1}
I20260806 11:40:34.970701 7280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:34.970170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:36681 (local address 127.21.14.195:59071)
0806 11:40:34.970498 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:34.970586 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:36681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":84,"threads_started":1}
I20260806 11:40:34.983299 7281 raft_consensus.cc:482] T 69fef14c78034517899675c07573120e P a44ebc4b518a42d5a327cfda159f9635 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 11:40:34.983474 7281 raft_consensus.cc:504] T 69fef14c78034517899675c07573120e P a44ebc4b518a42d5a327cfda159f9635 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558d5fa7ae6d48058c7e74f21bc12211" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 43875 } } peers { permanent_uuid: "9678ece9548c429ba9076df50c10b444" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 36077 } } peers { permanent_uuid: "a44ebc4b518a42d5a327cfda159f9635" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 39037 } }
I20260806 11:40:34.984081 7281 leader_election.cc:288] T 69fef14c78034517899675c07573120e P a44ebc4b518a42d5a327cfda159f9635 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 558d5fa7ae6d48058c7e74f21bc12211 (127.21.14.194:43875), 9678ece9548c429ba9076df50c10b444 (127.21.14.195:36077)
W20260806 11:40:34.985041 6828 leader_election.cc:341] T 69fef14c78034517899675c07573120e P a44ebc4b518a42d5a327cfda159f9635 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9678ece9548c429ba9076df50c10b444 (127.21.14.195:36077): Not found: Tablet not found: 69fef14c78034517899675c07573120e
W20260806 11:40:34.985148 6825 leader_election.cc:341] T 69fef14c78034517899675c07573120e P a44ebc4b518a42d5a327cfda159f9635 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 558d5fa7ae6d48058c7e74f21bc12211 (127.21.14.194:43875): Not found: Tablet not found: 69fef14c78034517899675c07573120e
I20260806 11:40:34.985206 6825 leader_election.cc:302] T 69fef14c78034517899675c07573120e P a44ebc4b518a42d5a327cfda159f9635 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a44ebc4b518a42d5a327cfda159f9635; no voters: 558d5fa7ae6d48058c7e74f21bc12211, 9678ece9548c429ba9076df50c10b444
I20260806 11:40:34.985365 7281 raft_consensus.cc:2726] T 69fef14c78034517899675c07573120e P a44ebc4b518a42d5a327cfda159f9635 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 11:40:35.561648 7282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:35.561146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:36681 (local address 127.21.14.193:51909)
0806 11:40:35.561468 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:35.561544 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:36681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":85,"threads_started":1}
I20260806 11:40:35.763893 7283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:35.763390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:36681 (local address 127.21.14.194:37767)
0806 11:40:35.763716 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:35.763785 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.254:36681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":93,"threads_started":1}
I20260806 11:40:35.945797 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 6765
I20260806 11:40:35.956244 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 6951
I20260806 11:40:35.964531 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 7090
2026-08-06T11:40:35Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (61035 ms)
[----------] 30 tests from SecurityITest (266889 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 11:40:36 dist-test-slave-7c35 krb5kdc[7286](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 11:40:36 dist-test-slave-7c35 krb5kdc[7286](info): set up 2 sockets
Aug 06 11:40:36 dist-test-slave-7c35 krb5kdc[7286](info): commencing operation
krb5kdc: starting...
W20260806 11:40:37.063855 21563 mini_kdc.cc:121] Time spent starting KDC: real 1.073s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Aug 06 11:40:37 dist-test-slave-7c35 krb5kdc[7286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016437, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T11:40:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T11:40:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.14.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab.
Entry for principal kudu/127.21.14.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab.
I20260806 11:40:37.222162 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:43459
--webserver_interface=127.21.14.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab
--principal=kudu/127.21.14.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:38267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.14.254:43459
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:40:37.345574 7301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:40:37.345904 7301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:40:37.345947 7301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:40:37.350263 7301 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 11:40:37.350344 7301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:40:37.350365 7301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:40:37.350382 7301 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 11:40:37.350399 7301 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 11:40:37.356150 7301 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:38267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.14.254:43459
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab
--principal=kudu/127.21.14.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:43459
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.14.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7301
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:40:37.357337 7301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:40:37.358402 7301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:40:37.368685 7307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:40:37.368674 7306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:40:37.368752 7301 server_base.cc:1034] running on GCE node
W20260806 11:40:37.368707 7309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:40:37.369798 7301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:40:37.371212 7301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:40:37.372406 7301 hybrid_clock.cc:648] HybridClock initialized: now 1786016437372373 us; error 50 us; skew 500 ppm
Aug 06 11:40:37 dist-test-slave-7c35 krb5kdc[7286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016437, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:40:37.375635 7301 init.cc:377] Logged in from keytab as kudu/127.21.14.254@KRBTEST.COM (short username kudu)
I20260806 11:40:37.376638 7301 webserver.cc:460] Webserver started at http://127.21.14.254:46223/ using document root <none> and password file <none>
I20260806 11:40:37.377241 7301 fs_manager.cc:359] Metadata directory not provided
I20260806 11:40:37.377307 7301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:40:37.377492 7301 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:40:37.379870 7301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/minicluster-data/master-0/data/instance:
uuid: "3d9d67076673479482aa861ff84140a8"
format_stamp: "Formatted at 2026-08-06 11:40:37 on dist-test-slave-7c35"
server_key: "4303d6dc65aefdfa5b62b2fcb46a02ac"
server_key_iv: "1bc9a8c2286de8a472ac4f5e9f190375"
server_key_version: "encryptionkey@0"
I20260806 11:40:37.380563 7301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/minicluster-data/master-0/wal/instance:
uuid: "3d9d67076673479482aa861ff84140a8"
format_stamp: "Formatted at 2026-08-06 11:40:37 on dist-test-slave-7c35"
server_key: "4303d6dc65aefdfa5b62b2fcb46a02ac"
server_key_iv: "1bc9a8c2286de8a472ac4f5e9f190375"
server_key_version: "encryptionkey@0"
I20260806 11:40:37.384836 7301 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 11:40:37.387926 7317 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:37.389413 7301 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 11:40:37.389601 7301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/minicluster-data/master-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/minicluster-data/master-0/wal
uuid: "3d9d67076673479482aa861ff84140a8"
format_stamp: "Formatted at 2026-08-06 11:40:37 on dist-test-slave-7c35"
server_key: "4303d6dc65aefdfa5b62b2fcb46a02ac"
server_key_iv: "1bc9a8c2286de8a472ac4f5e9f190375"
server_key_version: "encryptionkey@0"
I20260806 11:40:37.389750 7301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:40:37.413755 7301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:40:37.417294 7301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:40:37.417528 7301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:40:37.426039 7301 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.254:43459
I20260806 11:40:37.426043 7371 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.254:43459 every 8 connection(s)
I20260806 11:40:37.427294 7301 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
I20260806 11:40:37.430399 7372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:40:37.431408 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 7301
I20260806 11:40:37.431551 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/minicluster-data/master-0/wal/instance
I20260806 11:40:37.431869 21563 external_mini_cluster.cc:1428] Setting key 6929fcf64f84d7d0714898d69e402886
I20260806 11:40:37.437013 7372 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d9d67076673479482aa861ff84140a8: Bootstrap starting.
I20260806 11:40:37.439658 7372 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d9d67076673479482aa861ff84140a8: Neither blocks nor log segments found. Creating new log.
I20260806 11:40:37.440533 7372 log.cc:824] T 00000000000000000000000000000000 P 3d9d67076673479482aa861ff84140a8: Log is configured to *not* fsync() on all Append() calls
Aug 06 11:40:37 dist-test-slave-7c35 krb5kdc[7286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016437, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:40:37.443065 7372 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d9d67076673479482aa861ff84140a8: No bootstrap required, opened a new log
I20260806 11:40:37.448071 7376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:37.433482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45896 (local address 127.21.14.254:43459)
0806 11:40:37.433995 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:37.434011 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:37.434063 (+ 52us) server_negotiation.cc:407] Connection header received
0806 11:40:37.434734 (+ 671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:37.434753 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:37.435201 (+ 448us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:37.435556 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:40:37.436412 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:37.437471 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:37.438375 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:37.438966 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:37.439173 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:37.442056 (+ 2883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:40:37.442062 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:40:37.442078 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:40:37.442106 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:40:37.444265 (+ 2159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:40:37.444864 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:40:37.444870 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:40:37.444908 (+ 38us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:40:37.445041 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:40:37.445413 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:40:37.445419 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:40:37.445423 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:40:37.446098 (+ 675us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:40:37.446326 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:37.446824 (+ 498us) server_negotiation.cc:299] Negotiation successful
0806 11:40:37.447225 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":150,"threads_started":1}
I20260806 11:40:37.447983 7372 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d9d67076673479482aa861ff84140a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d9d67076673479482aa861ff84140a8" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 43459 } }
I20260806 11:40:37.448294 7372 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3d9d67076673479482aa861ff84140a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:40:37.448392 7372 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d9d67076673479482aa861ff84140a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d9d67076673479482aa861ff84140a8, State: Initialized, Role: FOLLOWER
I20260806 11:40:37.448915 7372 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d9d67076673479482aa861ff84140a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d9d67076673479482aa861ff84140a8" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 43459 } }
I20260806 11:40:37.449083 7372 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3d9d67076673479482aa861ff84140a8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 11:40:37.449152 7372 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d9d67076673479482aa861ff84140a8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 11:40:37.449252 7372 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d9d67076673479482aa861ff84140a8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:40:37.450356 7372 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d9d67076673479482aa861ff84140a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d9d67076673479482aa861ff84140a8" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 43459 } }
I20260806 11:40:37.450728 7372 leader_election.cc:302] T 00000000000000000000000000000000 P 3d9d67076673479482aa861ff84140a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3d9d67076673479482aa861ff84140a8; no voters:
I20260806 11:40:37.451089 7372 leader_election.cc:288] T 00000000000000000000000000000000 P 3d9d67076673479482aa861ff84140a8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 11:40:37.451246 7379 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d9d67076673479482aa861ff84140a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:40:37.451517 7379 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3d9d67076673479482aa861ff84140a8 [term 1 LEADER]: Becoming Leader. State: Replica: 3d9d67076673479482aa861ff84140a8, State: Running, Role: LEADER
I20260806 11:40:37.451870 7379 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d9d67076673479482aa861ff84140a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d9d67076673479482aa861ff84140a8" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 43459 } }
I20260806 11:40:37.452453 7372 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d9d67076673479482aa861ff84140a8 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 11:40:37.454061 7380 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d9d67076673479482aa861ff84140a8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3d9d67076673479482aa861ff84140a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d9d67076673479482aa861ff84140a8" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 43459 } } }
I20260806 11:40:37.454083 7381 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d9d67076673479482aa861ff84140a8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d9d67076673479482aa861ff84140a8. Latest consensus state: current_term: 1 leader_uuid: "3d9d67076673479482aa861ff84140a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d9d67076673479482aa861ff84140a8" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 43459 } } }
I20260806 11:40:37.454252 7381 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d9d67076673479482aa861ff84140a8 [sys.catalog]: This master's current role is: LEADER
I20260806 11:40:37.454252 7380 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d9d67076673479482aa861ff84140a8 [sys.catalog]: This master's current role is: LEADER
I20260806 11:40:37.454695 7388 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 11:40:37.457794 7388 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 11:40:37.463977 7388 catalog_manager.cc:1347] Generated new cluster ID: e670c608e0294f099ed635b1fafae351
I20260806 11:40:37.464063 7388 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 11:40:37.475651 7388 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 11:40:37.477031 7388 catalog_manager.cc:1504] Loading token signing keys...
I20260806 11:40:37.493484 7388 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3d9d67076673479482aa861ff84140a8: Generated new TSK 0
I20260806 11:40:37.494432 7388 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.14.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab.
Entry for principal kudu/127.21.14.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab.
I20260806 11:40:37.535856 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.193:0
--local_ip_for_outbound_sockets=127.21.14.193
--webserver_interface=127.21.14.193
--webserver_port=0
--tserver_master_addrs=127.21.14.254:43459
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab
--principal=kudu/127.21.14.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:38267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:40:37.680492 7400 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:40:37.680823 7400 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:40:37.680871 7400 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:40:37.685165 7400 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:40:37.685247 7400 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:40:37.685316 7400 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.193
I20260806 11:40:37.691323 7400 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:38267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab
--principal=kudu/127.21.14.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.14.193
--webserver_port=0
--tserver_master_addrs=127.21.14.254:43459
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7400
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.193
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:40:37.692508 7400 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:40:37.693651 7400 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:40:37.704393 7410 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:40:37.704406 7406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:40:37.704406 7408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:40:37.704784 7400 server_base.cc:1034] running on GCE node
I20260806 11:40:37.705253 7400 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:40:37.705921 7400 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:40:37.707113 7400 hybrid_clock.cc:648] HybridClock initialized: now 1786016437707078 us; error 45 us; skew 500 ppm
Aug 06 11:40:37 dist-test-slave-7c35 krb5kdc[7286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016437, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:40:37.710484 7400 init.cc:377] Logged in from keytab as kudu/127.21.14.193@KRBTEST.COM (short username kudu)
I20260806 11:40:37.711614 7400 webserver.cc:460] Webserver started at http://127.21.14.193:39589/ using document root <none> and password file <none>
I20260806 11:40:37.712250 7400 fs_manager.cc:359] Metadata directory not provided
I20260806 11:40:37.712352 7400 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:40:37.712615 7400 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:40:37.715188 7400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/minicluster-data/ts-0/data/instance:
uuid: "45084aa75dc24da09d757371c190e4ad"
format_stamp: "Formatted at 2026-08-06 11:40:37 on dist-test-slave-7c35"
server_key: "3966d082b62e1c93af2a5e09d6492fb4"
server_key_iv: "49262f8c6da96a95ba1c3cc4dd44a536"
server_key_version: "encryptionkey@0"
I20260806 11:40:37.715849 7400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/minicluster-data/ts-0/wal/instance:
uuid: "45084aa75dc24da09d757371c190e4ad"
format_stamp: "Formatted at 2026-08-06 11:40:37 on dist-test-slave-7c35"
server_key: "3966d082b62e1c93af2a5e09d6492fb4"
server_key_iv: "49262f8c6da96a95ba1c3cc4dd44a536"
server_key_version: "encryptionkey@0"
I20260806 11:40:37.720693 7400 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 11:40:37.724184 7417 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:37.725620 7400 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 11:40:37.725801 7400 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/minicluster-data/ts-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/minicluster-data/ts-0/wal
uuid: "45084aa75dc24da09d757371c190e4ad"
format_stamp: "Formatted at 2026-08-06 11:40:37 on dist-test-slave-7c35"
server_key: "3966d082b62e1c93af2a5e09d6492fb4"
server_key_iv: "49262f8c6da96a95ba1c3cc4dd44a536"
server_key_version: "encryptionkey@0"
I20260806 11:40:37.726047 7400 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:40:37.736148 7400 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:40:37.739670 7400 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:40:37.739907 7400 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:40:37.740561 7400 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:40:37.741459 7400 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:40:37.741534 7400 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:37.741609 7400 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:40:37.741663 7400 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:37.752382 7400 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.193:37191
I20260806 11:40:37.752413 7535 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.193:37191 every 8 connection(s)
I20260806 11:40:37.753477 7400 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
I20260806 11:40:37.755573 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 7400
I20260806 11:40:37.755740 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/minicluster-data/ts-0/wal/instance
I20260806 11:40:37.756093 21563 external_mini_cluster.cc:1428] Setting key 134cfaa89c0436b985007423fc63059e
Aug 06 11:40:37 dist-test-slave-7c35 krb5kdc[7286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016437, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.193@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.14.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.194@KRBTEST.COM" created.
I20260806 11:40:37.771343 7376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:37.755790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:60643 (local address 127.21.14.254:43459)
0806 11:40:37.755969 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:37.755973 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:37.757153 (+ 1180us) server_negotiation.cc:407] Connection header received
0806 11:40:37.759389 (+ 2236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:37.759393 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:37.759455 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:37.759542 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:40:37.761165 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:37.762043 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:37.763395 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:37.763861 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:37.764002 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:37.767067 (+ 3065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:40:37.767075 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:40:37.767077 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:40:37.767119 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:40:37.768719 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:40:37.769352 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:40:37.769355 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:40:37.769356 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:40:37.769440 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:40:37.769773 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:40:37.769776 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:40:37.769777 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:40:37.769966 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:40:37.770046 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:37.770702 (+ 656us) server_negotiation.cc:299] Negotiation successful
0806 11:40:37.771127 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 11:40:37.772301 7538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:37.756375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:43459 (local address 127.21.14.193:60643)
0806 11:40:37.756954 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:37.757036 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0806 11:40:37.759081 (+ 2045us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:40:37.759780 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:40:37.759793 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:40:37.760359 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:40:37.760974 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:37.760990 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:37.762192 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:37.762195 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:37.763251 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:37.763263 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:37.764088 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:37.764091 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:37.764359 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:37.765163 (+ 804us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:40:37.765182 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:40:37.766791 (+ 1609us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:40:37.768845 (+ 2054us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:40:37.768850 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:40:37.768865 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:40:37.769237 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:40:37.769526 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:40:37.769528 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:40:37.769529 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:40:37.769680 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:40:37.770055 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:40:37.770061 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:40:37.770448 (+ 387us) client_negotiation.cc:769] Sending connection context
0806 11:40:37.770625 (+ 177us) client_negotiation.cc:240] Negotiation successful
0806 11:40:37.770998 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":172,"threads_started":1}
I20260806 11:40:37.773859 7536 heartbeater.cc:347] Connected to a master server at 127.21.14.254:43459
I20260806 11:40:37.774183 7536 heartbeater.cc:464] Registering TS with master...
I20260806 11:40:37.774951 7536 heartbeater.cc:511] Master 127.21.14.254:43459 requested a full tablet report, sending...
I20260806 11:40:37.776731 7335 ts_manager.cc:194] Registered new tserver with Master: 45084aa75dc24da09d757371c190e4ad (127.21.14.193:37191)
I20260806 11:40:37.778075 7335 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.14.193@KRBTEST.COM'} at 127.21.14.193:60643
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab.
Entry for principal kudu/127.21.14.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab.
I20260806 11:40:37.790985 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.194:0
--local_ip_for_outbound_sockets=127.21.14.194
--webserver_interface=127.21.14.194
--webserver_port=0
--tserver_master_addrs=127.21.14.254:43459
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab
--principal=kudu/127.21.14.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:38267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:40:37.916673 7541 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:40:37.917016 7541 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:40:37.917098 7541 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:40:37.921384 7541 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:40:37.921499 7541 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:40:37.921595 7541 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.194
I20260806 11:40:37.927937 7541 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:38267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab
--principal=kudu/127.21.14.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.14.194
--webserver_port=0
--tserver_master_addrs=127.21.14.254:43459
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7541
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.194
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:40:37.929155 7541 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:40:37.930383 7541 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:40:37.940891 7548 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:40:37.940914 7549 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:40:37.940891 7551 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:40:37.941365 7541 server_base.cc:1034] running on GCE node
I20260806 11:40:37.941923 7541 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:40:37.942695 7541 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:40:37.943877 7541 hybrid_clock.cc:648] HybridClock initialized: now 1786016437943842 us; error 54 us; skew 500 ppm
Aug 06 11:40:37 dist-test-slave-7c35 krb5kdc[7286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016437, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:40:37.947031 7541 init.cc:377] Logged in from keytab as kudu/127.21.14.194@KRBTEST.COM (short username kudu)
I20260806 11:40:37.948165 7541 webserver.cc:460] Webserver started at http://127.21.14.194:34053/ using document root <none> and password file <none>
I20260806 11:40:37.948803 7541 fs_manager.cc:359] Metadata directory not provided
I20260806 11:40:37.948872 7541 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:40:37.949059 7541 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:40:37.951635 7541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/minicluster-data/ts-1/data/instance:
uuid: "1c8288a787f44388bd2d2304a3dcf1d6"
format_stamp: "Formatted at 2026-08-06 11:40:37 on dist-test-slave-7c35"
server_key: "0f1f44a411342a1ef5c058f91225959b"
server_key_iv: "ddba65f536a9aac3e59cace9fcc98ef5"
server_key_version: "encryptionkey@0"
I20260806 11:40:37.952329 7541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/minicluster-data/ts-1/wal/instance:
uuid: "1c8288a787f44388bd2d2304a3dcf1d6"
format_stamp: "Formatted at 2026-08-06 11:40:37 on dist-test-slave-7c35"
server_key: "0f1f44a411342a1ef5c058f91225959b"
server_key_iv: "ddba65f536a9aac3e59cace9fcc98ef5"
server_key_version: "encryptionkey@0"
I20260806 11:40:37.957468 7541 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 11:40:37.960711 7558 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:37.962296 7541 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 11:40:37.962493 7541 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/minicluster-data/ts-1/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/minicluster-data/ts-1/wal
uuid: "1c8288a787f44388bd2d2304a3dcf1d6"
format_stamp: "Formatted at 2026-08-06 11:40:37 on dist-test-slave-7c35"
server_key: "0f1f44a411342a1ef5c058f91225959b"
server_key_iv: "ddba65f536a9aac3e59cace9fcc98ef5"
server_key_version: "encryptionkey@0"
I20260806 11:40:37.962680 7541 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:40:37.996598 7541 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:40:38.000517 7541 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:40:38.000756 7541 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:40:38.001427 7541 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:40:38.002365 7541 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:40:38.002439 7541 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:38.002538 7541 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:40:38.002585 7541 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:38.012508 7541 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.194:36419
I20260806 11:40:38.012535 7681 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.194:36419 every 8 connection(s)
I20260806 11:40:38.013499 7541 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
I20260806 11:40:38.020035 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 7541
I20260806 11:40:38.020151 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/minicluster-data/ts-1/wal/instance
I20260806 11:40:38.020545 21563 external_mini_cluster.cc:1428] Setting key 25356e8e3b1e0034dfea72d3380fbfb1
Aug 06 11:40:38 dist-test-slave-7c35 krb5kdc[7286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016437, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.194@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:40:38.031594 7376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:38.015553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:43261 (local address 127.21.14.254:43459)
0806 11:40:38.015742 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:38.015746 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:38.016646 (+ 900us) server_negotiation.cc:407] Connection header received
0806 11:40:38.018104 (+ 1458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:38.018108 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:38.018173 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:38.018285 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:40:38.019585 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:38.020166 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:38.021903 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:38.022255 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:38.022395 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:38.026232 (+ 3837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:40:38.026239 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:40:38.026245 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:40:38.026282 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:40:38.028746 (+ 2464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:40:38.029434 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:40:38.029441 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:40:38.029444 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:40:38.029555 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:40:38.030127 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:40:38.030132 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:40:38.030133 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:40:38.030428 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:40:38.030551 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:38.031243 (+ 692us) server_negotiation.cc:299] Negotiation successful
0806 11:40:38.031408 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
WARNING: no policy specified for kudu/127.21.14.195@KRBTEST.COM; defaulting to no policy
I20260806 11:40:38.032629 7685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:38.015980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:43459 (local address 127.21.14.194:43261)
0806 11:40:38.016484 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:38.016529 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 11:40:38.017855 (+ 1326us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:40:38.018484 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:40:38.018491 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:40:38.018910 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:40:38.019422 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:38.019437 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:38.020347 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:38.020352 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:38.021762 (+ 1410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:38.021774 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:38.022493 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:38.022500 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:38.022724 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:38.023639 (+ 915us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:40:38.023670 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:40:38.025980 (+ 2310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:40:38.028911 (+ 2931us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:40:38.028919 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:40:38.028935 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:40:38.029288 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:40:38.029699 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:40:38.029701 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:40:38.029703 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:40:38.029878 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:40:38.030557 (+ 679us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:40:38.030564 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:40:38.030947 (+ 383us) client_negotiation.cc:769] Sending connection context
0806 11:40:38.031192 (+ 245us) client_negotiation.cc:240] Negotiation successful
0806 11:40:38.031627 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":125,"threads_started":1}
I20260806 11:40:38.034004 7683 heartbeater.cc:347] Connected to a master server at 127.21.14.254:43459
I20260806 11:40:38.034296 7683 heartbeater.cc:464] Registering TS with master...
I20260806 11:40:38.034970 7683 heartbeater.cc:511] Master 127.21.14.254:43459 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.195@KRBTEST.COM" created.
I20260806 11:40:38.036339 7335 ts_manager.cc:194] Registered new tserver with Master: 1c8288a787f44388bd2d2304a3dcf1d6 (127.21.14.194:36419)
I20260806 11:40:38.036981 7335 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.14.194@KRBTEST.COM'} at 127.21.14.194:43261
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab.
Entry for principal kudu/127.21.14.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab.
I20260806 11:40:38.054641 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.195:0
--local_ip_for_outbound_sockets=127.21.14.195
--webserver_interface=127.21.14.195
--webserver_port=0
--tserver_master_addrs=127.21.14.254:43459
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab
--principal=kudu/127.21.14.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:38267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:40:38.178694 7688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:40:38.179060 7688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:40:38.179180 7688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:40:38.183558 7688 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:40:38.183635 7688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:40:38.183699 7688 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.195
I20260806 11:40:38.189563 7688 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:38267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab
--principal=kudu/127.21.14.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.14.195
--webserver_port=0
--tserver_master_addrs=127.21.14.254:43459
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7688
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.195
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:40:38.190908 7688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:40:38.192075 7688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:40:38.204056 7694 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:40:38.204173 7693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:40:38.204056 7696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:40:38.204442 7688 server_base.cc:1034] running on GCE node
I20260806 11:40:38.204936 7688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:40:38.205674 7688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:40:38.206945 7688 hybrid_clock.cc:648] HybridClock initialized: now 1786016438206919 us; error 36 us; skew 500 ppm
Aug 06 11:40:38 dist-test-slave-7c35 krb5kdc[7286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016438, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:40:38.210276 7688 init.cc:377] Logged in from keytab as kudu/127.21.14.195@KRBTEST.COM (short username kudu)
I20260806 11:40:38.211539 7688 webserver.cc:460] Webserver started at http://127.21.14.195:41179/ using document root <none> and password file <none>
I20260806 11:40:38.212201 7688 fs_manager.cc:359] Metadata directory not provided
I20260806 11:40:38.212312 7688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:40:38.212586 7688 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:40:38.214936 7688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/minicluster-data/ts-2/data/instance:
uuid: "ff3dfbb2d97441b7948f7a049d12889a"
format_stamp: "Formatted at 2026-08-06 11:40:38 on dist-test-slave-7c35"
server_key: "ab2410bd974135f3ceba9993fd285888"
server_key_iv: "4f1ea4757a268019bfd4499f28845bd2"
server_key_version: "encryptionkey@0"
I20260806 11:40:38.215721 7688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/minicluster-data/ts-2/wal/instance:
uuid: "ff3dfbb2d97441b7948f7a049d12889a"
format_stamp: "Formatted at 2026-08-06 11:40:38 on dist-test-slave-7c35"
server_key: "ab2410bd974135f3ceba9993fd285888"
server_key_iv: "4f1ea4757a268019bfd4499f28845bd2"
server_key_version: "encryptionkey@0"
I20260806 11:40:38.221230 7688 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 11:40:38.224495 7704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:38.226086 7688 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 11:40:38.226217 7688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/minicluster-data/ts-2/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/minicluster-data/ts-2/wal
uuid: "ff3dfbb2d97441b7948f7a049d12889a"
format_stamp: "Formatted at 2026-08-06 11:40:38 on dist-test-slave-7c35"
server_key: "ab2410bd974135f3ceba9993fd285888"
server_key_iv: "4f1ea4757a268019bfd4499f28845bd2"
server_key_version: "encryptionkey@0"
I20260806 11:40:38.226387 7688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:40:38.249272 7688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:40:38.253000 7688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:40:38.253259 7688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:40:38.253979 7688 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:40:38.255043 7688 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:40:38.255119 7688 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:38.255196 7688 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:40:38.255240 7688 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:38.264958 7688 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.195:34077
I20260806 11:40:38.264991 7822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.195:34077 every 8 connection(s)
I20260806 11:40:38.266094 7688 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
I20260806 11:40:38.273820 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 7688
I20260806 11:40:38.274012 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/minicluster-data/ts-2/wal/instance
I20260806 11:40:38.274402 21563 external_mini_cluster.cc:1428] Setting key 810e3a97bd6b1fd9e490b3b9d70272a2
Aug 06 11:40:38 dist-test-slave-7c35 krb5kdc[7286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016438, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.195@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:40:38.281719 7376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:38.268359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:51025 (local address 127.21.14.254:43459)
0806 11:40:38.268557 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:38.268563 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:38.269547 (+ 984us) server_negotiation.cc:407] Connection header received
0806 11:40:38.271002 (+ 1455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:38.271006 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:38.271069 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:38.271152 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:40:38.272400 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:38.272960 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:38.274311 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:38.274616 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:38.274731 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:38.278010 (+ 3279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:40:38.278014 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:40:38.278016 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:40:38.278044 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:40:38.279629 (+ 1585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:40:38.280219 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:40:38.280222 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:40:38.280223 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:40:38.280317 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:40:38.280666 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:40:38.280668 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:40:38.280669 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:40:38.280863 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:40:38.280943 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:38.281474 (+ 531us) server_negotiation.cc:299] Negotiation successful
0806 11:40:38.281587 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260806 11:40:38.282685 7825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:38.268852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:43459 (local address 127.21.14.195:51025)
0806 11:40:38.269375 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:38.269421 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 11:40:38.270733 (+ 1312us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:40:38.271327 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:40:38.271335 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:40:38.271704 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:40:38.272229 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:38.272243 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:38.273116 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:38.273120 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:38.274180 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:38.274191 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:38.274787 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:38.274792 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:38.275047 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:38.275865 (+ 818us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:40:38.275888 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:40:38.277772 (+ 1884us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:40:38.279767 (+ 1995us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:40:38.279773 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:40:38.279789 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:40:38.280097 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:40:38.280420 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:40:38.280422 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:40:38.280424 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:40:38.280577 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:40:38.280957 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:40:38.280962 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:40:38.281232 (+ 270us) client_negotiation.cc:769] Sending connection context
0806 11:40:38.281409 (+ 177us) client_negotiation.cc:240] Negotiation successful
0806 11:40:38.281703 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":160,"threads_started":1}
I20260806 11:40:38.284060 7823 heartbeater.cc:347] Connected to a master server at 127.21.14.254:43459
I20260806 11:40:38.284340 7823 heartbeater.cc:464] Registering TS with master...
I20260806 11:40:38.284905 7823 heartbeater.cc:511] Master 127.21.14.254:43459 requested a full tablet report, sending...
I20260806 11:40:38.286083 7335 ts_manager.cc:194] Registered new tserver with Master: ff3dfbb2d97441b7948f7a049d12889a (127.21.14.195:34077)
I20260806 11:40:38.286705 7335 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.14.195@KRBTEST.COM'} at 127.21.14.195:51025
I20260806 11:40:38.288934 21563 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 11:40:38.299767 7376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:38.291231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59896 (local address 127.21.14.254:43459)
0806 11:40:38.291390 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:38.291395 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:38.291652 (+ 257us) server_negotiation.cc:407] Connection header received
0806 11:40:38.291821 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:38.291830 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:38.291877 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:38.291967 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:40:38.292735 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:38.293268 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:38.294171 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:38.294403 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:38.294525 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:38.296079 (+ 1554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:40:38.296089 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:40:38.296092 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:40:38.296118 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:40:38.297623 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:40:38.298064 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:40:38.298070 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:40:38.298072 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:40:38.298162 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:40:38.298456 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:40:38.298460 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:40:38.298461 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:40:38.298818 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:40:38.299007 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:38.299429 (+ 422us) server_negotiation.cc:299] Negotiation successful
0806 11:40:38.299585 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 11:40:38.303313 7335 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59896:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 11:40:38.306602 7335 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 11:40:38.323079 7839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:38.317110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48556 (local address 127.21.14.195:34077)
0806 11:40:38.317513 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:38.317526 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:38.317552 (+ 26us) server_negotiation.cc:407] Connection header received
0806 11:40:38.317638 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:38.317643 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:38.317891 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:38.318065 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:40:38.318354 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:38.319640 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:38.321620 (+ 1980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:38.322267 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:38.322415 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:38.322600 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:38.322752 (+ 152us) server_negotiation.cc:299] Negotiation successful
0806 11:40:38.322898 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":160,"threads_started":1}
I20260806 11:40:38.323081 7837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:38.316413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.195:34077 (local address 127.0.0.1:48556)
0806 11:40:38.317128 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:38.317190 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 11:40:38.317351 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:40:38.318046 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:40:38.318052 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:40:38.318077 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:40:38.318225 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:38.318232 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:38.319785 (+ 1553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:38.319788 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:38.321488 (+ 1700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:38.321497 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:38.322424 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:38.322428 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:38.322554 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:38.322597 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 11:40:38.322719 (+ 122us) client_negotiation.cc:240] Negotiation successful
0806 11:40:38.322911 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"mutex_wait_us":419,"spinlock_wait_cycles":2432,"thread_start_us":104,"threads_started":1}
I20260806 11:40:38.323127 7835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:38.316669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.193:37191 (local address 127.0.0.1:35962)
0806 11:40:38.317788 (+ 1119us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:38.317801 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 11:40:38.317890 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:40:38.318647 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:40:38.318651 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:40:38.318665 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:40:38.318775 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:38.318781 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:38.320148 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:38.320153 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:38.321233 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:38.321243 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:38.322493 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:38.322497 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:38.322603 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:38.322627 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 11:40:38.322698 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 11:40:38.322947 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1059,"mutex_wait_us":4,"thread_start_us":105,"threads_started":1}
I20260806 11:40:38.323540 7834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:38.316260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.194:36419 (local address 127.0.0.1:40986)
0806 11:40:38.317723 (+ 1463us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:38.317737 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 11:40:38.317828 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:40:38.318514 (+ 686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:40:38.318517 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:40:38.318532 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:40:38.318650 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:38.318656 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:38.319726 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:38.319733 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:38.321298 (+ 1565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:38.321308 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:38.322698 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:38.322702 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:38.322784 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:38.322807 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 11:40:38.322947 (+ 140us) client_negotiation.cc:240] Negotiation successful
0806 11:40:38.323021 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":667,"mutex_wait_us":53,"thread_start_us":83,"threads_started":1}
I20260806 11:40:38.323843 7838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:38.316998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35962 (local address 127.21.14.193:37191)
0806 11:40:38.318185 (+ 1187us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:38.318195 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:38.318217 (+ 22us) server_negotiation.cc:407] Connection header received
0806 11:40:38.318318 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:38.318323 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:38.318505 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:38.318653 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:40:38.318936 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:38.320008 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:38.321407 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:38.322335 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:38.323331 (+ 996us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:38.323509 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:38.323566 (+ 57us) server_negotiation.cc:299] Negotiation successful
0806 11:40:38.323675 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1076,"thread_start_us":105,"threads_started":1}
I20260806 11:40:38.323410 7836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:38.316465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40986 (local address 127.21.14.194:36419)
0806 11:40:38.317643 (+ 1178us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:38.317659 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:38.317949 (+ 290us) server_negotiation.cc:407] Connection header received
0806 11:40:38.318041 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:38.318047 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:38.318224 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:38.318367 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:40:38.318786 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:38.319589 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:38.321452 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:38.322335 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:38.322468 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:38.322631 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:38.323146 (+ 515us) server_negotiation.cc:299] Negotiation successful
0806 11:40:38.323264 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1077,"thread_start_us":121,"threads_started":1}
I20260806 11:40:38.326771 7612 tablet_service.cc:1467] Processing CreateTablet for tablet 71eeca1dd39f42fbafd6824c36450af9 (DEFAULT_TABLE table=test-table [id=ed2144b1c7134d9596456b854e62be5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:40:38.326773 7755 tablet_service.cc:1467] Processing CreateTablet for tablet 71eeca1dd39f42fbafd6824c36450af9 (DEFAULT_TABLE table=test-table [id=ed2144b1c7134d9596456b854e62be5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:40:38.328037 7755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71eeca1dd39f42fbafd6824c36450af9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:40:38.328037 7612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71eeca1dd39f42fbafd6824c36450af9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:40:38.329152 7467 tablet_service.cc:1467] Processing CreateTablet for tablet 71eeca1dd39f42fbafd6824c36450af9 (DEFAULT_TABLE table=test-table [id=ed2144b1c7134d9596456b854e62be5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:40:38.330320 7467 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71eeca1dd39f42fbafd6824c36450af9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:40:38.335577 7841 tablet_bootstrap.cc:492] T 71eeca1dd39f42fbafd6824c36450af9 P ff3dfbb2d97441b7948f7a049d12889a: Bootstrap starting.
I20260806 11:40:38.336834 7840 tablet_bootstrap.cc:492] T 71eeca1dd39f42fbafd6824c36450af9 P 1c8288a787f44388bd2d2304a3dcf1d6: Bootstrap starting.
I20260806 11:40:38.337795 7842 tablet_bootstrap.cc:492] T 71eeca1dd39f42fbafd6824c36450af9 P 45084aa75dc24da09d757371c190e4ad: Bootstrap starting.
I20260806 11:40:38.338819 7841 tablet_bootstrap.cc:654] T 71eeca1dd39f42fbafd6824c36450af9 P ff3dfbb2d97441b7948f7a049d12889a: Neither blocks nor log segments found. Creating new log.
I20260806 11:40:38.339017 7840 tablet_bootstrap.cc:654] T 71eeca1dd39f42fbafd6824c36450af9 P 1c8288a787f44388bd2d2304a3dcf1d6: Neither blocks nor log segments found. Creating new log.
I20260806 11:40:38.340070 7842 tablet_bootstrap.cc:654] T 71eeca1dd39f42fbafd6824c36450af9 P 45084aa75dc24da09d757371c190e4ad: Neither blocks nor log segments found. Creating new log.
I20260806 11:40:38.340512 7841 log.cc:824] T 71eeca1dd39f42fbafd6824c36450af9 P ff3dfbb2d97441b7948f7a049d12889a: Log is configured to *not* fsync() on all Append() calls
I20260806 11:40:38.340512 7840 log.cc:824] T 71eeca1dd39f42fbafd6824c36450af9 P 1c8288a787f44388bd2d2304a3dcf1d6: Log is configured to *not* fsync() on all Append() calls
I20260806 11:40:38.340858 7842 log.cc:824] T 71eeca1dd39f42fbafd6824c36450af9 P 45084aa75dc24da09d757371c190e4ad: Log is configured to *not* fsync() on all Append() calls
I20260806 11:40:38.343022 7841 tablet_bootstrap.cc:492] T 71eeca1dd39f42fbafd6824c36450af9 P ff3dfbb2d97441b7948f7a049d12889a: No bootstrap required, opened a new log
I20260806 11:40:38.343022 7840 tablet_bootstrap.cc:492] T 71eeca1dd39f42fbafd6824c36450af9 P 1c8288a787f44388bd2d2304a3dcf1d6: No bootstrap required, opened a new log
I20260806 11:40:38.343246 7841 ts_tablet_manager.cc:1397] T 71eeca1dd39f42fbafd6824c36450af9 P ff3dfbb2d97441b7948f7a049d12889a: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260806 11:40:38.343246 7840 ts_tablet_manager.cc:1397] T 71eeca1dd39f42fbafd6824c36450af9 P 1c8288a787f44388bd2d2304a3dcf1d6: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.004s
I20260806 11:40:38.344255 7842 tablet_bootstrap.cc:492] T 71eeca1dd39f42fbafd6824c36450af9 P 45084aa75dc24da09d757371c190e4ad: No bootstrap required, opened a new log
I20260806 11:40:38.344399 7842 ts_tablet_manager.cc:1397] T 71eeca1dd39f42fbafd6824c36450af9 P 45084aa75dc24da09d757371c190e4ad: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260806 11:40:38.347667 7840 raft_consensus.cc:348] T 71eeca1dd39f42fbafd6824c36450af9 P 1c8288a787f44388bd2d2304a3dcf1d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c8288a787f44388bd2d2304a3dcf1d6" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 36419 } } peers { permanent_uuid: "ff3dfbb2d97441b7948f7a049d12889a" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 34077 } } peers { permanent_uuid: "45084aa75dc24da09d757371c190e4ad" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 37191 } }
I20260806 11:40:38.347667 7841 raft_consensus.cc:348] T 71eeca1dd39f42fbafd6824c36450af9 P ff3dfbb2d97441b7948f7a049d12889a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c8288a787f44388bd2d2304a3dcf1d6" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 36419 } } peers { permanent_uuid: "ff3dfbb2d97441b7948f7a049d12889a" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 34077 } } peers { permanent_uuid: "45084aa75dc24da09d757371c190e4ad" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 37191 } }
I20260806 11:40:38.348006 7840 raft_consensus.cc:374] T 71eeca1dd39f42fbafd6824c36450af9 P 1c8288a787f44388bd2d2304a3dcf1d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:40:38.348006 7841 raft_consensus.cc:374] T 71eeca1dd39f42fbafd6824c36450af9 P ff3dfbb2d97441b7948f7a049d12889a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:40:38.348090 7841 raft_consensus.cc:729] T 71eeca1dd39f42fbafd6824c36450af9 P ff3dfbb2d97441b7948f7a049d12889a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff3dfbb2d97441b7948f7a049d12889a, State: Initialized, Role: FOLLOWER
I20260806 11:40:38.348111 7840 raft_consensus.cc:729] T 71eeca1dd39f42fbafd6824c36450af9 P 1c8288a787f44388bd2d2304a3dcf1d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c8288a787f44388bd2d2304a3dcf1d6, State: Initialized, Role: FOLLOWER
I20260806 11:40:38.348657 7841 consensus_queue.cc:260] T 71eeca1dd39f42fbafd6824c36450af9 P ff3dfbb2d97441b7948f7a049d12889a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c8288a787f44388bd2d2304a3dcf1d6" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 36419 } } peers { permanent_uuid: "ff3dfbb2d97441b7948f7a049d12889a" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 34077 } } peers { permanent_uuid: "45084aa75dc24da09d757371c190e4ad" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 37191 } }
I20260806 11:40:38.348666 7840 consensus_queue.cc:260] T 71eeca1dd39f42fbafd6824c36450af9 P 1c8288a787f44388bd2d2304a3dcf1d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c8288a787f44388bd2d2304a3dcf1d6" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 36419 } } peers { permanent_uuid: "ff3dfbb2d97441b7948f7a049d12889a" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 34077 } } peers { permanent_uuid: "45084aa75dc24da09d757371c190e4ad" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 37191 } }
I20260806 11:40:38.348713 7842 raft_consensus.cc:348] T 71eeca1dd39f42fbafd6824c36450af9 P 45084aa75dc24da09d757371c190e4ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c8288a787f44388bd2d2304a3dcf1d6" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 36419 } } peers { permanent_uuid: "ff3dfbb2d97441b7948f7a049d12889a" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 34077 } } peers { permanent_uuid: "45084aa75dc24da09d757371c190e4ad" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 37191 } }
I20260806 11:40:38.349045 7842 raft_consensus.cc:374] T 71eeca1dd39f42fbafd6824c36450af9 P 45084aa75dc24da09d757371c190e4ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:40:38.349180 7842 raft_consensus.cc:729] T 71eeca1dd39f42fbafd6824c36450af9 P 45084aa75dc24da09d757371c190e4ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45084aa75dc24da09d757371c190e4ad, State: Initialized, Role: FOLLOWER
I20260806 11:40:38.349539 7823 heartbeater.cc:503] Master 127.21.14.254:43459 was elected leader, sending a full tablet report...
I20260806 11:40:38.349812 7683 heartbeater.cc:503] Master 127.21.14.254:43459 was elected leader, sending a full tablet report...
I20260806 11:40:38.349747 7842 consensus_queue.cc:260] T 71eeca1dd39f42fbafd6824c36450af9 P 45084aa75dc24da09d757371c190e4ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c8288a787f44388bd2d2304a3dcf1d6" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 36419 } } peers { permanent_uuid: "ff3dfbb2d97441b7948f7a049d12889a" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 34077 } } peers { permanent_uuid: "45084aa75dc24da09d757371c190e4ad" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 37191 } }
I20260806 11:40:38.350504 7841 ts_tablet_manager.cc:1428] T 71eeca1dd39f42fbafd6824c36450af9 P ff3dfbb2d97441b7948f7a049d12889a: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260806 11:40:38.350780 7842 ts_tablet_manager.cc:1428] T 71eeca1dd39f42fbafd6824c36450af9 P 45084aa75dc24da09d757371c190e4ad: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 11:40:38.350978 7840 ts_tablet_manager.cc:1428] T 71eeca1dd39f42fbafd6824c36450af9 P 1c8288a787f44388bd2d2304a3dcf1d6: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260806 11:40:38.351270 7536 heartbeater.cc:503] Master 127.21.14.254:43459 was elected leader, sending a full tablet report...
I20260806 11:40:38.368978 7846 raft_consensus.cc:482] T 71eeca1dd39f42fbafd6824c36450af9 P 1c8288a787f44388bd2d2304a3dcf1d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 11:40:38.369266 7846 raft_consensus.cc:504] T 71eeca1dd39f42fbafd6824c36450af9 P 1c8288a787f44388bd2d2304a3dcf1d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c8288a787f44388bd2d2304a3dcf1d6" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 36419 } } peers { permanent_uuid: "ff3dfbb2d97441b7948f7a049d12889a" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 34077 } } peers { permanent_uuid: "45084aa75dc24da09d757371c190e4ad" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 37191 } }
I20260806 11:40:38.370507 7846 leader_election.cc:288] T 71eeca1dd39f42fbafd6824c36450af9 P 1c8288a787f44388bd2d2304a3dcf1d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff3dfbb2d97441b7948f7a049d12889a (127.21.14.195:34077), 45084aa75dc24da09d757371c190e4ad (127.21.14.193:37191)
I20260806 11:40:38.374953 7839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:38.370914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:40745 (local address 127.21.14.195:34077)
0806 11:40:38.371080 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:38.371085 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:38.371105 (+ 20us) server_negotiation.cc:407] Connection header received
0806 11:40:38.371293 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:38.371297 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:38.371365 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:38.371448 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:40:38.371775 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:38.372512 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:38.373810 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:38.374263 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:38.374367 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:38.374447 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:38.374687 (+ 240us) server_negotiation.cc:299] Negotiation successful
0806 11:40:38.374778 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 11:40:38.374989 7849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:38.370800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.193:37191 (local address 127.21.14.194:42907)
0806 11:40:38.371270 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:38.371283 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 11:40:38.371390 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:40:38.371702 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:40:38.371705 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:40:38.371741 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:40:38.371865 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:38.371872 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:38.372837 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:38.372840 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:38.373724 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:38.373764 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:38.374478 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:38.374482 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:38.374597 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:38.374616 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 11:40:38.374717 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 11:40:38.374783 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":384,"mutex_wait_us":44,"thread_start_us":147,"threads_started":1}
I20260806 11:40:38.374991 7838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:38.370977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:42907 (local address 127.21.14.193:37191)
0806 11:40:38.371115 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:38.371120 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:38.371377 (+ 257us) server_negotiation.cc:407] Connection header received
0806 11:40:38.371503 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:38.371506 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:38.371591 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:38.371683 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:40:38.372110 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:38.372689 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:38.373888 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:38.374340 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:38.374462 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:38.374520 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:38.374755 (+ 235us) server_negotiation.cc:299] Negotiation successful
0806 11:40:38.374869 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 11:40:38.374991 7685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:38.370770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.195:34077 (local address 127.21.14.194:40745)
0806 11:40:38.371019 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:38.371047 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 11:40:38.371159 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:40:38.371491 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:40:38.371494 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:40:38.371513 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:40:38.371657 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:38.371668 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:38.372732 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:38.372738 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:38.373669 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:38.373679 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:38.374407 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:38.374414 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:38.374530 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:38.374567 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 11:40:38.374640 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 11:40:38.374739 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"mutex_wait_us":1}
I20260806 11:40:38.375828 7489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71eeca1dd39f42fbafd6824c36450af9" candidate_uuid: "1c8288a787f44388bd2d2304a3dcf1d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45084aa75dc24da09d757371c190e4ad" is_pre_election: true
I20260806 11:40:38.375917 7775 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71eeca1dd39f42fbafd6824c36450af9" candidate_uuid: "1c8288a787f44388bd2d2304a3dcf1d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff3dfbb2d97441b7948f7a049d12889a" is_pre_election: true
I20260806 11:40:38.376209 7775 raft_consensus.cc:2454] T 71eeca1dd39f42fbafd6824c36450af9 P ff3dfbb2d97441b7948f7a049d12889a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c8288a787f44388bd2d2304a3dcf1d6 in term 0.
I20260806 11:40:38.376209 7489 raft_consensus.cc:2454] T 71eeca1dd39f42fbafd6824c36450af9 P 45084aa75dc24da09d757371c190e4ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c8288a787f44388bd2d2304a3dcf1d6 in term 0.
I20260806 11:40:38.376828 7559 leader_election.cc:302] T 71eeca1dd39f42fbafd6824c36450af9 P 1c8288a787f44388bd2d2304a3dcf1d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1c8288a787f44388bd2d2304a3dcf1d6, 45084aa75dc24da09d757371c190e4ad, ff3dfbb2d97441b7948f7a049d12889a; no voters:
I20260806 11:40:38.377192 7846 raft_consensus.cc:2781] T 71eeca1dd39f42fbafd6824c36450af9 P 1c8288a787f44388bd2d2304a3dcf1d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 11:40:38.377301 7846 raft_consensus.cc:482] T 71eeca1dd39f42fbafd6824c36450af9 P 1c8288a787f44388bd2d2304a3dcf1d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 11:40:38.377393 7846 raft_consensus.cc:3037] T 71eeca1dd39f42fbafd6824c36450af9 P 1c8288a787f44388bd2d2304a3dcf1d6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:40:38.378715 7846 raft_consensus.cc:504] T 71eeca1dd39f42fbafd6824c36450af9 P 1c8288a787f44388bd2d2304a3dcf1d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c8288a787f44388bd2d2304a3dcf1d6" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 36419 } } peers { permanent_uuid: "ff3dfbb2d97441b7948f7a049d12889a" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 34077 } } peers { permanent_uuid: "45084aa75dc24da09d757371c190e4ad" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 37191 } }
I20260806 11:40:38.379177 7846 leader_election.cc:288] T 71eeca1dd39f42fbafd6824c36450af9 P 1c8288a787f44388bd2d2304a3dcf1d6 [CANDIDATE]: Term 1 election: Requested vote from peers ff3dfbb2d97441b7948f7a049d12889a (127.21.14.195:34077), 45084aa75dc24da09d757371c190e4ad (127.21.14.193:37191)
I20260806 11:40:38.379521 7775 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71eeca1dd39f42fbafd6824c36450af9" candidate_uuid: "1c8288a787f44388bd2d2304a3dcf1d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff3dfbb2d97441b7948f7a049d12889a"
I20260806 11:40:38.379647 7489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71eeca1dd39f42fbafd6824c36450af9" candidate_uuid: "1c8288a787f44388bd2d2304a3dcf1d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45084aa75dc24da09d757371c190e4ad"
I20260806 11:40:38.379675 7775 raft_consensus.cc:3037] T 71eeca1dd39f42fbafd6824c36450af9 P ff3dfbb2d97441b7948f7a049d12889a [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:40:38.379786 7489 raft_consensus.cc:3037] T 71eeca1dd39f42fbafd6824c36450af9 P 45084aa75dc24da09d757371c190e4ad [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:40:38.381142 7489 raft_consensus.cc:2454] T 71eeca1dd39f42fbafd6824c36450af9 P 45084aa75dc24da09d757371c190e4ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c8288a787f44388bd2d2304a3dcf1d6 in term 1.
I20260806 11:40:38.381142 7775 raft_consensus.cc:2454] T 71eeca1dd39f42fbafd6824c36450af9 P ff3dfbb2d97441b7948f7a049d12889a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c8288a787f44388bd2d2304a3dcf1d6 in term 1.
I20260806 11:40:38.381531 7559 leader_election.cc:302] T 71eeca1dd39f42fbafd6824c36450af9 P 1c8288a787f44388bd2d2304a3dcf1d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c8288a787f44388bd2d2304a3dcf1d6, ff3dfbb2d97441b7948f7a049d12889a; no voters:
I20260806 11:40:38.381765 7846 raft_consensus.cc:2781] T 71eeca1dd39f42fbafd6824c36450af9 P 1c8288a787f44388bd2d2304a3dcf1d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:40:38.381991 7846 raft_consensus.cc:686] T 71eeca1dd39f42fbafd6824c36450af9 P 1c8288a787f44388bd2d2304a3dcf1d6 [term 1 LEADER]: Becoming Leader. State: Replica: 1c8288a787f44388bd2d2304a3dcf1d6, State: Running, Role: LEADER
I20260806 11:40:38.382398 7846 consensus_queue.cc:237] T 71eeca1dd39f42fbafd6824c36450af9 P 1c8288a787f44388bd2d2304a3dcf1d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c8288a787f44388bd2d2304a3dcf1d6" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 36419 } } peers { permanent_uuid: "ff3dfbb2d97441b7948f7a049d12889a" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 34077 } } peers { permanent_uuid: "45084aa75dc24da09d757371c190e4ad" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 37191 } }
I20260806 11:40:38.385856 7335 catalog_manager.cc:5495] T 71eeca1dd39f42fbafd6824c36450af9 P 1c8288a787f44388bd2d2304a3dcf1d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1c8288a787f44388bd2d2304a3dcf1d6 (127.21.14.194). New cstate: current_term: 1 leader_uuid: "1c8288a787f44388bd2d2304a3dcf1d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c8288a787f44388bd2d2304a3dcf1d6" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 36419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff3dfbb2d97441b7948f7a049d12889a" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 34077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45084aa75dc24da09d757371c190e4ad" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 37191 } health_report { overall_health: UNKNOWN } } }
I20260806 11:40:38.404120 7836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:38.400192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40994 (local address 127.21.14.194:36419)
0806 11:40:38.400346 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:38.400353 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:38.400373 (+ 20us) server_negotiation.cc:407] Connection header received
0806 11:40:38.400473 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:38.400475 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:38.400537 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:38.400680 (+ 143us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 11:40:38.400970 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:38.401490 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:38.402394 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:38.402632 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:38.402752 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:38.403170 (+ 418us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 11:40:38.403665 (+ 495us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 11:40:38.403815 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:38.403858 (+ 43us) server_negotiation.cc:299] Negotiation successful
0806 11:40:38.403926 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 11:40:38.411175 7489 raft_consensus.cc:1261] T 71eeca1dd39f42fbafd6824c36450af9 P 45084aa75dc24da09d757371c190e4ad [term 1 FOLLOWER]: Refusing update from remote peer 1c8288a787f44388bd2d2304a3dcf1d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 11:40:38.411175 7775 raft_consensus.cc:1261] T 71eeca1dd39f42fbafd6824c36450af9 P ff3dfbb2d97441b7948f7a049d12889a [term 1 FOLLOWER]: Refusing update from remote peer 1c8288a787f44388bd2d2304a3dcf1d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 11:40:38.411978 7846 consensus_queue.cc:1035] T 71eeca1dd39f42fbafd6824c36450af9 P 1c8288a787f44388bd2d2304a3dcf1d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45084aa75dc24da09d757371c190e4ad" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 37191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 11:40:38.412213 7851 consensus_queue.cc:1035] T 71eeca1dd39f42fbafd6824c36450af9 P 1c8288a787f44388bd2d2304a3dcf1d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff3dfbb2d97441b7948f7a049d12889a" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 34077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 11:40:38.421444 7857 mvcc.cc:205] Tried to move back new op lower bound from 7315523331722371072 to 7315523331617513472. Current Snapshot: MvccSnapshot[applied={T|T < 7315523331722371072}]
I20260806 11:40:38.420802 7855 mvcc.cc:205] Tried to move back new op lower bound from 7315523331722371072 to 7315523331617513472. Current Snapshot: MvccSnapshot[applied={T|T < 7315523331722371072}]
I20260806 11:40:38.431767 7335 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59896:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 11:40:38.431975 7335 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59896:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 11:40:38.435523 7335 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3d9d67076673479482aa861ff84140a8: Sending DeleteTablet for 3 replicas of tablet 71eeca1dd39f42fbafd6824c36450af9
I20260806 11:40:38.436712 7755 tablet_service.cc:1514] Processing DeleteTablet for tablet 71eeca1dd39f42fbafd6824c36450af9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:40:38 UTC) from {username='kudu', principal='kudu/127.21.14.254@KRBTEST.COM'} at 127.0.0.1:48556
I20260806 11:40:38.436717 7612 tablet_service.cc:1514] Processing DeleteTablet for tablet 71eeca1dd39f42fbafd6824c36450af9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:40:38 UTC) from {username='kudu', principal='kudu/127.21.14.254@KRBTEST.COM'} at 127.0.0.1:40986
I20260806 11:40:38.437825 7467 tablet_service.cc:1514] Processing DeleteTablet for tablet 71eeca1dd39f42fbafd6824c36450af9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:40:38 UTC) from {username='kudu', principal='kudu/127.21.14.254@KRBTEST.COM'} at 127.0.0.1:35962
I20260806 11:40:38.438002 7867 tablet_replica.cc:330] T 71eeca1dd39f42fbafd6824c36450af9 P ff3dfbb2d97441b7948f7a049d12889a: stopping tablet replica
I20260806 11:40:38.438382 7868 tablet_replica.cc:330] T 71eeca1dd39f42fbafd6824c36450af9 P 45084aa75dc24da09d757371c190e4ad: stopping tablet replica
I20260806 11:40:38.438402 7867 raft_consensus.cc:2229] T 71eeca1dd39f42fbafd6824c36450af9 P ff3dfbb2d97441b7948f7a049d12889a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 11:40:38.438622 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 7400
I20260806 11:40:38.438808 7867 raft_consensus.cc:2258] T 71eeca1dd39f42fbafd6824c36450af9 P ff3dfbb2d97441b7948f7a049d12889a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 11:40:38.441500 7867 ts_tablet_manager.cc:1905] T 71eeca1dd39f42fbafd6824c36450af9 P ff3dfbb2d97441b7948f7a049d12889a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 11:40:38.445482 7867 ts_tablet_manager.cc:1918] T 71eeca1dd39f42fbafd6824c36450af9 P ff3dfbb2d97441b7948f7a049d12889a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 11:40:38.445628 7867 log.cc:1196] T 71eeca1dd39f42fbafd6824c36450af9 P ff3dfbb2d97441b7948f7a049d12889a: Deleting WAL directory at /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786016169045816-21563-0/minicluster-data/ts-2/wal/wals/71eeca1dd39f42fbafd6824c36450af9
I20260806 11:40:38.446063 7867 ts_tablet_manager.cc:1939] T 71eeca1dd39f42fbafd6824c36450af9 P ff3dfbb2d97441b7948f7a049d12889a: Deleting consensus metadata
I20260806 11:40:38.447680 7319 catalog_manager.cc:4841] TS ff3dfbb2d97441b7948f7a049d12889a (127.21.14.195:34077): tablet 71eeca1dd39f42fbafd6824c36450af9 (table test-table [id=ed2144b1c7134d9596456b854e62be5e]) successfully deleted
W20260806 11:40:38.449200 7322 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.14.193:37191 (error 108)
I20260806 11:40:38.449360 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 7541
I20260806 11:40:38.455188 7834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:38.454690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.193:37191 (local address 127.0.0.1:35974)
0806 11:40:38.454913 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:38.455050 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.193:37191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260806 11:40:38.455426 7322 catalog_manager.cc:4584] TS 45084aa75dc24da09d757371c190e4ad (127.21.14.193:37191): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 71eeca1dd39f42fbafd6824c36450af9: Network error: Client connection negotiation failed: client connection to 127.21.14.193:37191: connect: Connection refused (error 111)
I20260806 11:40:38.459429 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 7688
I20260806 11:40:38.462560 7834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:38.459698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.194:36419 (local address 127.0.0.1:41004)
0806 11:40:38.462373 (+ 2675us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:38.462460 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.194:36419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2538}
I20260806 11:40:38.467850 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 7301
2026-08-06T11:40:38Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2503 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-06T11:40:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T11:40:38Z Disabled control of system clock
I20260806 11:40:38.505239 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786016169045816-21563-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786016169045816-21563-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786016169045816-21563-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:33591
--webserver_interface=127.21.14.254
--webserver_port=0
--builtin_ntp_servers=127.21.14.212:45305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.14.254:33591
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 11:40:38.626008 7873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:40:38.626341 7873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:40:38.626380 7873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:40:38.630781 7873 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 11:40:38.630929 7873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:40:38.630951 7873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:40:38.630970 7873 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 11:40:38.630985 7873 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 11:40:38.636301 7873 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:45305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786016169045816-21563-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786016169045816-21563-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.14.254:33591
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:33591
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.14.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7873
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786016169045816-21563-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:40:38.637579 7873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:40:38.639181 7873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:40:38.648758 7881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:40:38.648772 7878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:40:38.648768 7879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:40:38.648823 7873 server_base.cc:1034] running on GCE node
I20260806 11:40:38.649716 7873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:40:38.650986 7873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:40:38.652212 7873 hybrid_clock.cc:648] HybridClock initialized: now 1786016438652174 us; error 56 us; skew 500 ppm
I20260806 11:40:38.654345 7873 webserver.cc:460] Webserver started at http://127.21.14.254:45855/ using document root <none> and password file <none>
I20260806 11:40:38.655082 7873 fs_manager.cc:359] Metadata directory not provided
I20260806 11:40:38.655181 7873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:40:38.655444 7873 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:40:38.657747 7873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786016169045816-21563-0/minicluster-data/master-0/data/instance:
uuid: "325b0a599c3e4d5b956553f99520cef3"
format_stamp: "Formatted at 2026-08-06 11:40:38 on dist-test-slave-7c35"
server_key: "660c452af346b6490ec8d1babf0e6378"
server_key_iv: "fd37d5335c1fe62aeda1d0a0a9f07193"
server_key_version: "encryptionkey@0"
I20260806 11:40:38.658403 7873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786016169045816-21563-0/minicluster-data/master-0/wal/instance:
uuid: "325b0a599c3e4d5b956553f99520cef3"
format_stamp: "Formatted at 2026-08-06 11:40:38 on dist-test-slave-7c35"
server_key: "660c452af346b6490ec8d1babf0e6378"
server_key_iv: "fd37d5335c1fe62aeda1d0a0a9f07193"
server_key_version: "encryptionkey@0"
I20260806 11:40:38.662734 7873 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260806 11:40:38.665550 7887 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:38.667047 7873 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 11:40:38.667222 7873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786016169045816-21563-0/minicluster-data/master-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786016169045816-21563-0/minicluster-data/master-0/wal
uuid: "325b0a599c3e4d5b956553f99520cef3"
format_stamp: "Formatted at 2026-08-06 11:40:38 on dist-test-slave-7c35"
server_key: "660c452af346b6490ec8d1babf0e6378"
server_key_iv: "fd37d5335c1fe62aeda1d0a0a9f07193"
server_key_version: "encryptionkey@0"
I20260806 11:40:38.667359 7873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786016169045816-21563-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786016169045816-21563-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786016169045816-21563-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:40:38.693656 7873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:40:38.694504 7873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:40:38.694741 7873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:40:38.702942 7873 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.254:33591
I20260806 11:40:38.702982 7944 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.254:33591 every 8 connection(s)
I20260806 11:40:38.704121 7873 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
I20260806 11:40:38.707626 7945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:40:38.713465 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 7873
I20260806 11:40:38.713680 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786016169045816-21563-0/minicluster-data/master-0/wal/instance
I20260806 11:40:38.713970 21563 external_mini_cluster.cc:1428] Setting key 4c266f00d96c9c6324e2fb9095244952
I20260806 11:40:38.714006 7945 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 325b0a599c3e4d5b956553f99520cef3: Bootstrap starting.
I20260806 11:40:38.718026 7945 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 325b0a599c3e4d5b956553f99520cef3: Neither blocks nor log segments found. Creating new log.
I20260806 11:40:38.719009 7945 log.cc:824] T 00000000000000000000000000000000 P 325b0a599c3e4d5b956553f99520cef3: Log is configured to *not* fsync() on all Append() calls
I20260806 11:40:38.721550 7945 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 325b0a599c3e4d5b956553f99520cef3: No bootstrap required, opened a new log
I20260806 11:40:38.724797 7948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:38.715503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56188 (local address 127.21.14.254:33591)
0806 11:40:38.716155 (+ 652us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:38.716171 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:38.716208 (+ 37us) server_negotiation.cc:407] Connection header received
0806 11:40:38.716913 (+ 705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:38.716932 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:38.717532 (+ 600us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:38.717870 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:40:38.718341 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:38.719513 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:38.720805 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:38.721308 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:38.721490 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:38.722556 (+ 1066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:40:38.722562 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:40:38.722576 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 11:40:38.723101 (+ 525us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:40:38.723169 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 11:40:38.723193 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:40:38.723285 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:38.723489 (+ 204us) server_negotiation.cc:299] Negotiation successful
0806 11:40:38.723819 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":114,"threads_started":1}
I20260806 11:40:38.725431 7945 raft_consensus.cc:348] T 00000000000000000000000000000000 P 325b0a599c3e4d5b956553f99520cef3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "325b0a599c3e4d5b956553f99520cef3" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 33591 } }
I20260806 11:40:38.725735 7945 raft_consensus.cc:374] T 00000000000000000000000000000000 P 325b0a599c3e4d5b956553f99520cef3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:40:38.725835 7945 raft_consensus.cc:729] T 00000000000000000000000000000000 P 325b0a599c3e4d5b956553f99520cef3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 325b0a599c3e4d5b956553f99520cef3, State: Initialized, Role: FOLLOWER
I20260806 11:40:38.726370 7945 consensus_queue.cc:260] T 00000000000000000000000000000000 P 325b0a599c3e4d5b956553f99520cef3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "325b0a599c3e4d5b956553f99520cef3" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 33591 } }
I20260806 11:40:38.726642 7945 raft_consensus.cc:388] T 00000000000000000000000000000000 P 325b0a599c3e4d5b956553f99520cef3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 11:40:38.726754 7945 raft_consensus.cc:482] T 00000000000000000000000000000000 P 325b0a599c3e4d5b956553f99520cef3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 11:40:38.726891 7945 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 325b0a599c3e4d5b956553f99520cef3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:40:38.728204 7945 raft_consensus.cc:504] T 00000000000000000000000000000000 P 325b0a599c3e4d5b956553f99520cef3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "325b0a599c3e4d5b956553f99520cef3" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 33591 } }
I20260806 11:40:38.728552 7945 leader_election.cc:302] T 00000000000000000000000000000000 P 325b0a599c3e4d5b956553f99520cef3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 325b0a599c3e4d5b956553f99520cef3; no voters:
I20260806 11:40:38.728925 7945 leader_election.cc:288] T 00000000000000000000000000000000 P 325b0a599c3e4d5b956553f99520cef3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 11:40:38.729103 7950 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 325b0a599c3e4d5b956553f99520cef3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:40:38.729369 7950 raft_consensus.cc:686] T 00000000000000000000000000000000 P 325b0a599c3e4d5b956553f99520cef3 [term 1 LEADER]: Becoming Leader. State: Replica: 325b0a599c3e4d5b956553f99520cef3, State: Running, Role: LEADER
I20260806 11:40:38.729709 7950 consensus_queue.cc:237] T 00000000000000000000000000000000 P 325b0a599c3e4d5b956553f99520cef3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "325b0a599c3e4d5b956553f99520cef3" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 33591 } }
I20260806 11:40:38.730502 7945 sys_catalog.cc:564] T 00000000000000000000000000000000 P 325b0a599c3e4d5b956553f99520cef3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 11:40:38.731617 7952 sys_catalog.cc:455] T 00000000000000000000000000000000 P 325b0a599c3e4d5b956553f99520cef3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 325b0a599c3e4d5b956553f99520cef3. Latest consensus state: current_term: 1 leader_uuid: "325b0a599c3e4d5b956553f99520cef3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "325b0a599c3e4d5b956553f99520cef3" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 33591 } } }
I20260806 11:40:38.731791 7952 sys_catalog.cc:458] T 00000000000000000000000000000000 P 325b0a599c3e4d5b956553f99520cef3 [sys.catalog]: This master's current role is: LEADER
I20260806 11:40:38.731634 7951 sys_catalog.cc:455] T 00000000000000000000000000000000 P 325b0a599c3e4d5b956553f99520cef3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "325b0a599c3e4d5b956553f99520cef3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "325b0a599c3e4d5b956553f99520cef3" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 33591 } } }
I20260806 11:40:38.731882 7951 sys_catalog.cc:458] T 00000000000000000000000000000000 P 325b0a599c3e4d5b956553f99520cef3 [sys.catalog]: This master's current role is: LEADER
I20260806 11:40:38.733230 7959 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 11:40:38.736377 7959 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 11:40:38.743316 7959 catalog_manager.cc:1347] Generated new cluster ID: 4358ba6c475d4d0dbfa884f6fd49cea3
I20260806 11:40:38.743441 7959 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 11:40:38.753098 7959 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 11:40:38.754382 7959 catalog_manager.cc:1504] Loading token signing keys...
I20260806 11:40:38.759822 7959 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 325b0a599c3e4d5b956553f99520cef3: Generated new TSK 0
I20260806 11:40:38.760643 7959 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 11:40:38.766356 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786016169045816-21563-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786016169045816-21563-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786016169045816-21563-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.193:0
--local_ip_for_outbound_sockets=127.21.14.193
--webserver_interface=127.21.14.193
--webserver_port=0
--tserver_master_addrs=127.21.14.254:33591
--builtin_ntp_servers=127.21.14.212:45305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 11:40:38.899022 7969 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:40:38.899369 7969 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:40:38.899451 7969 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:40:38.903975 7969 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:40:38.904064 7969 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:40:38.904158 7969 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.193
I20260806 11:40:38.909250 7969 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:45305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786016169045816-21563-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786016169045816-21563-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.14.193
--webserver_port=0
--tserver_master_addrs=127.21.14.254:33591
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7969
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.193
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786016169045816-21563-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:40:38.910537 7969 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:40:38.912075 7969 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:40:38.922223 7974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:40:38.922223 7975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:40:38.922276 7977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:40:38.922704 7969 server_base.cc:1034] running on GCE node
I20260806 11:40:38.923210 7969 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:40:38.923959 7969 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:40:38.925171 7969 hybrid_clock.cc:648] HybridClock initialized: now 1786016438925158 us; error 28 us; skew 500 ppm
I20260806 11:40:38.927249 7969 webserver.cc:460] Webserver started at http://127.21.14.193:35471/ using document root <none> and password file <none>
I20260806 11:40:38.927883 7969 fs_manager.cc:359] Metadata directory not provided
I20260806 11:40:38.927950 7969 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:40:38.928189 7969 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:40:38.930570 7969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786016169045816-21563-0/minicluster-data/ts-0/data/instance:
uuid: "2364216a2f364a2aa8bdf7a6219778f5"
format_stamp: "Formatted at 2026-08-06 11:40:38 on dist-test-slave-7c35"
server_key: "3304937f444f0b54ad2499f2a16c0e8b"
server_key_iv: "73c8ec74a1b69e11f45d4107d2949849"
server_key_version: "encryptionkey@0"
I20260806 11:40:38.931214 7969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786016169045816-21563-0/minicluster-data/ts-0/wal/instance:
uuid: "2364216a2f364a2aa8bdf7a6219778f5"
format_stamp: "Formatted at 2026-08-06 11:40:38 on dist-test-slave-7c35"
server_key: "3304937f444f0b54ad2499f2a16c0e8b"
server_key_iv: "73c8ec74a1b69e11f45d4107d2949849"
server_key_version: "encryptionkey@0"
I20260806 11:40:38.935491 7969 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 11:40:38.938295 7983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:38.939812 7969 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 11:40:38.939970 7969 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786016169045816-21563-0/minicluster-data/ts-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786016169045816-21563-0/minicluster-data/ts-0/wal
uuid: "2364216a2f364a2aa8bdf7a6219778f5"
format_stamp: "Formatted at 2026-08-06 11:40:38 on dist-test-slave-7c35"
server_key: "3304937f444f0b54ad2499f2a16c0e8b"
server_key_iv: "73c8ec74a1b69e11f45d4107d2949849"
server_key_version: "encryptionkey@0"
I20260806 11:40:38.940124 7969 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786016169045816-21563-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786016169045816-21563-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786016169045816-21563-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:40:38.950484 7969 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:40:38.951344 7969 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:40:38.951561 7969 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:40:38.952207 7969 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:40:38.953229 7969 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:40:38.953302 7969 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:38.953378 7969 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:40:38.953431 7969 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:38.963685 7969 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.193:38947
I20260806 11:40:38.963716 8102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.193:38947 every 8 connection(s)
I20260806 11:40:38.964710 7969 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
I20260806 11:40:38.964995 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 7969
I20260806 11:40:38.965144 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786016169045816-21563-0/minicluster-data/ts-0/wal/instance
I20260806 11:40:38.965545 21563 external_mini_cluster.cc:1428] Setting key 192eb9556e65217e870eb3d88b4624a1
I20260806 11:40:38.968075 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786016169045816-21563-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786016169045816-21563-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786016169045816-21563-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.194:0
--local_ip_for_outbound_sockets=127.21.14.194
--webserver_interface=127.21.14.194
--webserver_port=0
--tserver_master_addrs=127.21.14.254:33591
--builtin_ntp_servers=127.21.14.212:45305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 11:40:38.977257 7948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:38.966890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:46873 (local address 127.21.14.254:33591)
0806 11:40:38.967029 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:38.967033 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:38.968435 (+ 1402us) server_negotiation.cc:407] Connection header received
0806 11:40:38.970556 (+ 2121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:38.970559 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:38.970622 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:38.970751 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:40:38.972045 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:38.972748 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:38.974762 (+ 2014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:38.975058 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:38.975168 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:38.976720 (+ 1552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:40:38.976723 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:40:38.976726 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 11:40:38.976741 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:40:38.976760 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 11:40:38.976768 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:40:38.976829 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:38.977054 (+ 225us) server_negotiation.cc:299] Negotiation successful
0806 11:40:38.977129 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 11:40:38.978762 8105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:38.967386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:33591 (local address 127.21.14.193:46873)
0806 11:40:38.968224 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:38.968291 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0806 11:40:38.970310 (+ 2019us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:40:38.970983 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:40:38.970991 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:40:38.971123 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:40:38.971864 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:38.971888 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:38.972905 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:38.972912 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:38.974606 (+ 1694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:38.974622 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:38.975270 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:38.975277 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:38.975558 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:38.976328 (+ 770us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 11:40:38.976332 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:40:38.976405 (+ 73us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 11:40:38.976516 (+ 111us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:40:38.976864 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:40:38.976878 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:40:38.976887 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 11:40:38.977053 (+ 166us) client_negotiation.cc:240] Negotiation successful
0806 11:40:38.977452 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":119,"threads_started":1}
I20260806 11:40:38.980463 8103 heartbeater.cc:347] Connected to a master server at 127.21.14.254:33591
I20260806 11:40:38.980757 8103 heartbeater.cc:464] Registering TS with master...
I20260806 11:40:38.981437 8103 heartbeater.cc:511] Master 127.21.14.254:33591 requested a full tablet report, sending...
I20260806 11:40:38.983232 7907 ts_manager.cc:194] Registered new tserver with Master: 2364216a2f364a2aa8bdf7a6219778f5 (127.21.14.193:38947)
I20260806 11:40:38.984490 7907 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.14.193:46873
W20260806 11:40:39.091476 8106 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:40:39.091837 8106 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:40:39.091921 8106 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:40:39.096253 8106 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:40:39.096359 8106 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:40:39.096454 8106 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.194
I20260806 11:40:39.101828 8106 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:45305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786016169045816-21563-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786016169045816-21563-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.14.194
--webserver_port=0
--tserver_master_addrs=127.21.14.254:33591
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8106
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.194
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786016169045816-21563-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:40:39.103132 8106 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:40:39.104687 8106 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:40:39.115217 8114 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:40:39.115327 8111 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:40:39.115235 8112 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:40:39.115474 8106 server_base.cc:1034] running on GCE node
I20260806 11:40:39.116119 8106 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:40:39.116874 8106 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:40:39.118068 8106 hybrid_clock.cc:648] HybridClock initialized: now 1786016439118049 us; error 36 us; skew 500 ppm
I20260806 11:40:39.120095 8106 webserver.cc:460] Webserver started at http://127.21.14.194:40369/ using document root <none> and password file <none>
I20260806 11:40:39.120728 8106 fs_manager.cc:359] Metadata directory not provided
I20260806 11:40:39.120793 8106 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:40:39.121021 8106 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:40:39.123355 8106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786016169045816-21563-0/minicluster-data/ts-1/data/instance:
uuid: "a34a15c7b1064fd1a9c09c774f7484e4"
format_stamp: "Formatted at 2026-08-06 11:40:39 on dist-test-slave-7c35"
server_key: "c6a66540d762e7eb3492a9a37c2e05ff"
server_key_iv: "2ae22b67b6c54aadc0ab9bb541a0070f"
server_key_version: "encryptionkey@0"
I20260806 11:40:39.123996 8106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786016169045816-21563-0/minicluster-data/ts-1/wal/instance:
uuid: "a34a15c7b1064fd1a9c09c774f7484e4"
format_stamp: "Formatted at 2026-08-06 11:40:39 on dist-test-slave-7c35"
server_key: "c6a66540d762e7eb3492a9a37c2e05ff"
server_key_iv: "2ae22b67b6c54aadc0ab9bb541a0070f"
server_key_version: "encryptionkey@0"
I20260806 11:40:39.128084 8106 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260806 11:40:39.131119 8120 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:39.132612 8106 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 11:40:39.132731 8106 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786016169045816-21563-0/minicluster-data/ts-1/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786016169045816-21563-0/minicluster-data/ts-1/wal
uuid: "a34a15c7b1064fd1a9c09c774f7484e4"
format_stamp: "Formatted at 2026-08-06 11:40:39 on dist-test-slave-7c35"
server_key: "c6a66540d762e7eb3492a9a37c2e05ff"
server_key_iv: "2ae22b67b6c54aadc0ab9bb541a0070f"
server_key_version: "encryptionkey@0"
I20260806 11:40:39.132860 8106 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786016169045816-21563-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786016169045816-21563-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786016169045816-21563-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:40:39.163404 8106 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:40:39.164161 8106 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:40:39.164335 8106 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:40:39.165004 8106 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:40:39.166086 8106 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:40:39.166165 8106 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:39.166235 8106 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:40:39.166282 8106 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:39.176381 8106 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.194:37371
I20260806 11:40:39.176435 8245 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.194:37371 every 8 connection(s)
I20260806 11:40:39.177430 8106 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
I20260806 11:40:39.187472 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 8106
I20260806 11:40:39.187597 7948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:39.179582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:52373 (local address 127.21.14.254:33591)
0806 11:40:39.179729 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:39.179733 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:39.180835 (+ 1102us) server_negotiation.cc:407] Connection header received
0806 11:40:39.182347 (+ 1512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:39.182351 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:39.182418 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:39.182535 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:40:39.183614 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:39.184166 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:39.185411 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:39.185655 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:39.185778 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:39.187032 (+ 1254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:40:39.187036 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:40:39.187039 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 11:40:39.187061 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:40:39.187082 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 11:40:39.187092 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:40:39.187174 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:39.187355 (+ 181us) server_negotiation.cc:299] Negotiation successful
0806 11:40:39.187429 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 11:40:39.187659 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786016169045816-21563-0/minicluster-data/ts-1/wal/instance
I20260806 11:40:39.187984 21563 external_mini_cluster.cc:1428] Setting key ec8c4f6afd48cdc11eb8838956042fd5
I20260806 11:40:39.188711 8248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:39.179962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:33591 (local address 127.21.14.194:52373)
0806 11:40:39.180664 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:39.180709 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 11:40:39.182101 (+ 1392us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:40:39.182723 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:40:39.182730 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:40:39.182824 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:40:39.183449 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:39.183465 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:39.184322 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:39.184326 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:39.185263 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:39.185274 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:39.185836 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:39.185841 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:39.186036 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:39.186691 (+ 655us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 11:40:39.186693 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:40:39.186739 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 11:40:39.186813 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:40:39.187179 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:40:39.187191 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:40:39.187199 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 11:40:39.187343 (+ 144us) client_negotiation.cc:240] Negotiation successful
0806 11:40:39.187672 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":117,"threads_started":1}
I20260806 11:40:39.190182 8246 heartbeater.cc:347] Connected to a master server at 127.21.14.254:33591
I20260806 11:40:39.190503 8246 heartbeater.cc:464] Registering TS with master...
I20260806 11:40:39.190568 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786016169045816-21563-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786016169045816-21563-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786016169045816-21563-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.195:0
--local_ip_for_outbound_sockets=127.21.14.195
--webserver_interface=127.21.14.195
--webserver_port=0
--tserver_master_addrs=127.21.14.254:33591
--builtin_ntp_servers=127.21.14.212:45305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 11:40:39.191202 8246 heartbeater.cc:511] Master 127.21.14.254:33591 requested a full tablet report, sending...
I20260806 11:40:39.192505 7906 ts_manager.cc:194] Registered new tserver with Master: a34a15c7b1064fd1a9c09c774f7484e4 (127.21.14.194:37371)
I20260806 11:40:39.193173 7906 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.14.194:52373
W20260806 11:40:39.313622 8249 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:40:39.313918 8249 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:40:39.313961 8249 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:40:39.318243 8249 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:40:39.318317 8249 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:40:39.318377 8249 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.195
I20260806 11:40:39.323603 8249 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:45305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786016169045816-21563-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786016169045816-21563-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.14.195
--webserver_port=0
--tserver_master_addrs=127.21.14.254:33591
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8249
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.195
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786016169045816-21563-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:40:39.324813 8249 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:40:39.326311 8249 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:40:39.337318 8254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:40:39.337318 8257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:40:39.337327 8255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:40:39.337399 8249 server_base.cc:1034] running on GCE node
I20260806 11:40:39.338244 8249 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:40:39.339090 8249 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:40:39.340298 8249 hybrid_clock.cc:648] HybridClock initialized: now 1786016439340258 us; error 56 us; skew 500 ppm
I20260806 11:40:39.342408 8249 webserver.cc:460] Webserver started at http://127.21.14.195:45631/ using document root <none> and password file <none>
I20260806 11:40:39.343173 8249 fs_manager.cc:359] Metadata directory not provided
I20260806 11:40:39.343281 8249 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:40:39.343552 8249 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:40:39.346094 8249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786016169045816-21563-0/minicluster-data/ts-2/data/instance:
uuid: "045ec27f837840c0a95e4f0431c52ef9"
format_stamp: "Formatted at 2026-08-06 11:40:39 on dist-test-slave-7c35"
server_key: "688d88f36c7f32c1707069d9d985cc8a"
server_key_iv: "95256d49a8d4f84fd89d47227a4c1acb"
server_key_version: "encryptionkey@0"
I20260806 11:40:39.346743 8249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786016169045816-21563-0/minicluster-data/ts-2/wal/instance:
uuid: "045ec27f837840c0a95e4f0431c52ef9"
format_stamp: "Formatted at 2026-08-06 11:40:39 on dist-test-slave-7c35"
server_key: "688d88f36c7f32c1707069d9d985cc8a"
server_key_iv: "95256d49a8d4f84fd89d47227a4c1acb"
server_key_version: "encryptionkey@0"
I20260806 11:40:39.351226 8249 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 11:40:39.354171 8263 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:39.355710 8249 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 11:40:39.355895 8249 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786016169045816-21563-0/minicluster-data/ts-2/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786016169045816-21563-0/minicluster-data/ts-2/wal
uuid: "045ec27f837840c0a95e4f0431c52ef9"
format_stamp: "Formatted at 2026-08-06 11:40:39 on dist-test-slave-7c35"
server_key: "688d88f36c7f32c1707069d9d985cc8a"
server_key_iv: "95256d49a8d4f84fd89d47227a4c1acb"
server_key_version: "encryptionkey@0"
I20260806 11:40:39.356089 8249 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786016169045816-21563-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786016169045816-21563-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786016169045816-21563-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:40:39.377482 8249 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:40:39.378307 8249 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:40:39.378536 8249 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:40:39.379350 8249 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:40:39.380422 8249 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:40:39.380499 8249 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:39.380570 8249 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:40:39.380605 8249 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:39.389922 8249 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.195:35177
I20260806 11:40:39.389982 8383 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.195:35177 every 8 connection(s)
I20260806 11:40:39.391033 8249 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
I20260806 11:40:39.399550 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 8249
I20260806 11:40:39.399720 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786016169045816-21563-0/minicluster-data/ts-2/wal/instance
I20260806 11:40:39.400028 21563 external_mini_cluster.cc:1428] Setting key 42a7a2d9465518eb5a5a43f3f3afe6a0
I20260806 11:40:39.401566 7948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:39.393268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:56495 (local address 127.21.14.254:33591)
0806 11:40:39.393451 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:39.393456 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:39.394605 (+ 1149us) server_negotiation.cc:407] Connection header received
0806 11:40:39.396202 (+ 1597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:39.396206 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:39.396272 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:39.396358 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:40:39.397338 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:39.397895 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:39.399139 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:39.399374 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:39.399529 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:39.400936 (+ 1407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:40:39.400940 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:40:39.400944 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 11:40:39.400960 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:40:39.400984 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 11:40:39.400994 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:40:39.401094 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:39.401327 (+ 233us) server_negotiation.cc:299] Negotiation successful
0806 11:40:39.401397 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260806 11:40:39.402648 8386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:39.393687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:33591 (local address 127.21.14.195:56495)
0806 11:40:39.394421 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:39.394465 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 11:40:39.395930 (+ 1465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:40:39.396556 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:40:39.396585 (+ 29us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:40:39.396678 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:40:39.397202 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:39.397215 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:39.398015 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:39.398018 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:39.399019 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:39.399029 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:39.399602 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:39.399609 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:39.399840 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:39.400604 (+ 764us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 11:40:39.400608 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:40:39.400669 (+ 61us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 11:40:39.400759 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:40:39.401126 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:40:39.401141 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:40:39.401147 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 11:40:39.401292 (+ 145us) client_negotiation.cc:240] Negotiation successful
0806 11:40:39.401627 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":129,"threads_started":1}
I20260806 11:40:39.403896 8384 heartbeater.cc:347] Connected to a master server at 127.21.14.254:33591
I20260806 11:40:39.404119 8384 heartbeater.cc:464] Registering TS with master...
I20260806 11:40:39.404626 8384 heartbeater.cc:511] Master 127.21.14.254:33591 requested a full tablet report, sending...
I20260806 11:40:39.405647 7906 ts_manager.cc:194] Registered new tserver with Master: 045ec27f837840c0a95e4f0431c52ef9 (127.21.14.195:35177)
I20260806 11:40:39.406275 7906 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.14.195:56495
I20260806 11:40:39.414193 21563 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 11:40:39.420290 7948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:39.416229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56194 (local address 127.21.14.254:33591)
0806 11:40:39.416354 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:39.416358 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:39.416532 (+ 174us) server_negotiation.cc:407] Connection header received
0806 11:40:39.416660 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:39.416662 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:39.416705 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:39.416774 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:40:39.417046 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:39.417561 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:39.418418 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:39.418664 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:39.418757 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:39.419761 (+ 1004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:40:39.419765 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:40:39.419767 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 11:40:39.419780 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:40:39.419800 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 11:40:39.419808 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:40:39.419886 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:39.420101 (+ 215us) server_negotiation.cc:299] Negotiation successful
0806 11:40:39.420169 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 11:40:39.423910 7906 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56194:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 11:40:39.427245 7906 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 11:40:39.444000 8396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:39.437058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.195:35177 (local address 127.0.0.1:58766)
0806 11:40:39.437993 (+ 935us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:39.438056 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0806 11:40:39.438205 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:40:39.439224 (+ 1019us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:40:39.439305 (+ 81us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:40:39.439344 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:40:39.439540 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:39.439550 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:39.440745 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:39.440749 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:39.442081 (+ 1332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:39.442091 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:39.443346 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:39.443349 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:39.443475 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:39.443517 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 11:40:39.443651 (+ 134us) client_negotiation.cc:240] Negotiation successful
0806 11:40:39.443808 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":737,"mutex_wait_us":4,"thread_start_us":110,"threads_started":1}
I20260806 11:40:39.444026 8397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:39.437216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58766 (local address 127.21.14.195:35177)
0806 11:40:39.438597 (+ 1381us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:39.438612 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:39.438641 (+ 29us) server_negotiation.cc:407] Connection header received
0806 11:40:39.438730 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:39.438737 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:39.439018 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:39.439242 (+ 224us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:40:39.439709 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:39.440613 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:39.442216 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:39.442923 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:39.443044 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:39.443273 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:39.443713 (+ 440us) server_negotiation.cc:299] Negotiation successful
0806 11:40:39.443836 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1268,"thread_start_us":177,"threads_started":1}
I20260806 11:40:39.444195 8400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:39.438080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.193:38947 (local address 127.0.0.1:57424)
0806 11:40:39.438951 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:39.438964 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 11:40:39.439068 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:40:39.439747 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:40:39.439750 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:40:39.439772 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:40:39.439868 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:39.439874 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:39.441276 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:39.441279 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:39.442602 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:39.442614 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:39.443812 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:39.443815 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:39.443911 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:39.443932 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 11:40:39.443993 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 11:40:39.444073 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":802,"mutex_wait_us":41,"thread_start_us":82,"threads_started":1}
I20260806 11:40:39.445089 8401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:39.438460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57424 (local address 127.21.14.193:38947)
0806 11:40:39.438729 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:39.438736 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:39.439065 (+ 329us) server_negotiation.cc:407] Connection header received
0806 11:40:39.439251 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:39.439257 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:39.439441 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:39.439630 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:40:39.440001 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:39.441158 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:39.442757 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:39.443629 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:39.443725 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:39.444683 (+ 958us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:39.444741 (+ 58us) server_negotiation.cc:299] Negotiation successful
0806 11:40:39.444930 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":84,"threads_started":1}
I20260806 11:40:39.445318 8398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:39.437302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.194:37371 (local address 127.0.0.1:56860)
0806 11:40:39.438251 (+ 949us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:39.438265 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 11:40:39.438342 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:40:39.439526 (+ 1184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:40:39.439532 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:40:39.439557 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:40:39.439664 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:39.439671 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:39.441158 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:39.441162 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:39.442602 (+ 1440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:39.442614 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:39.444047 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:39.444054 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:39.444802 (+ 748us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:39.444829 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 11:40:39.444906 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 11:40:39.444977 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":884,"mutex_wait_us":149,"thread_start_us":159,"threads_started":1}
I20260806 11:40:39.447121 8399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:39.437875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56860 (local address 127.21.14.194:37371)
0806 11:40:39.438916 (+ 1041us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:39.438929 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:39.438956 (+ 27us) server_negotiation.cc:407] Connection header received
0806 11:40:39.439043 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:39.439049 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:39.439354 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:39.439559 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:40:39.439805 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:39.441010 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:39.442760 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:39.443685 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:39.443791 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:39.443971 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:39.445663 (+ 1692us) server_negotiation.cc:299] Negotiation successful
0806 11:40:39.446971 (+ 1308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":884,"thread_start_us":140,"threads_started":1}
I20260806 11:40:39.448127 8313 tablet_service.cc:1467] Processing CreateTablet for tablet 60e2d83e507e453e8890063093fe12d8 (DEFAULT_TABLE table=test-table [id=eac309b543af47b49efede8d6ed1e074]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:40:39.448127 8032 tablet_service.cc:1467] Processing CreateTablet for tablet 60e2d83e507e453e8890063093fe12d8 (DEFAULT_TABLE table=test-table [id=eac309b543af47b49efede8d6ed1e074]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:40:39.449378 8313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60e2d83e507e453e8890063093fe12d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:40:39.449378 8032 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60e2d83e507e453e8890063093fe12d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:40:39.450073 8172 tablet_service.cc:1467] Processing CreateTablet for tablet 60e2d83e507e453e8890063093fe12d8 (DEFAULT_TABLE table=test-table [id=eac309b543af47b49efede8d6ed1e074]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:40:39.451100 8172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60e2d83e507e453e8890063093fe12d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:40:39.457561 8402 tablet_bootstrap.cc:492] T 60e2d83e507e453e8890063093fe12d8 P a34a15c7b1064fd1a9c09c774f7484e4: Bootstrap starting.
I20260806 11:40:39.458554 8404 tablet_bootstrap.cc:492] T 60e2d83e507e453e8890063093fe12d8 P 2364216a2f364a2aa8bdf7a6219778f5: Bootstrap starting.
I20260806 11:40:39.459245 8403 tablet_bootstrap.cc:492] T 60e2d83e507e453e8890063093fe12d8 P 045ec27f837840c0a95e4f0431c52ef9: Bootstrap starting.
I20260806 11:40:39.460922 8404 tablet_bootstrap.cc:654] T 60e2d83e507e453e8890063093fe12d8 P 2364216a2f364a2aa8bdf7a6219778f5: Neither blocks nor log segments found. Creating new log.
I20260806 11:40:39.460942 8402 tablet_bootstrap.cc:654] T 60e2d83e507e453e8890063093fe12d8 P a34a15c7b1064fd1a9c09c774f7484e4: Neither blocks nor log segments found. Creating new log.
I20260806 11:40:39.461437 8403 tablet_bootstrap.cc:654] T 60e2d83e507e453e8890063093fe12d8 P 045ec27f837840c0a95e4f0431c52ef9: Neither blocks nor log segments found. Creating new log.
I20260806 11:40:39.461891 8404 log.cc:824] T 60e2d83e507e453e8890063093fe12d8 P 2364216a2f364a2aa8bdf7a6219778f5: Log is configured to *not* fsync() on all Append() calls
I20260806 11:40:39.461921 8402 log.cc:824] T 60e2d83e507e453e8890063093fe12d8 P a34a15c7b1064fd1a9c09c774f7484e4: Log is configured to *not* fsync() on all Append() calls
I20260806 11:40:39.462244 8403 log.cc:824] T 60e2d83e507e453e8890063093fe12d8 P 045ec27f837840c0a95e4f0431c52ef9: Log is configured to *not* fsync() on all Append() calls
I20260806 11:40:39.464080 8402 tablet_bootstrap.cc:492] T 60e2d83e507e453e8890063093fe12d8 P a34a15c7b1064fd1a9c09c774f7484e4: No bootstrap required, opened a new log
I20260806 11:40:39.464083 8404 tablet_bootstrap.cc:492] T 60e2d83e507e453e8890063093fe12d8 P 2364216a2f364a2aa8bdf7a6219778f5: No bootstrap required, opened a new log
I20260806 11:40:39.464313 8404 ts_tablet_manager.cc:1397] T 60e2d83e507e453e8890063093fe12d8 P 2364216a2f364a2aa8bdf7a6219778f5: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 11:40:39.464318 8402 ts_tablet_manager.cc:1397] T 60e2d83e507e453e8890063093fe12d8 P a34a15c7b1064fd1a9c09c774f7484e4: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260806 11:40:39.468232 8404 raft_consensus.cc:348] T 60e2d83e507e453e8890063093fe12d8 P 2364216a2f364a2aa8bdf7a6219778f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "045ec27f837840c0a95e4f0431c52ef9" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 35177 } } peers { permanent_uuid: "a34a15c7b1064fd1a9c09c774f7484e4" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37371 } } peers { permanent_uuid: "2364216a2f364a2aa8bdf7a6219778f5" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 38947 } }
I20260806 11:40:39.468237 8402 raft_consensus.cc:348] T 60e2d83e507e453e8890063093fe12d8 P a34a15c7b1064fd1a9c09c774f7484e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "045ec27f837840c0a95e4f0431c52ef9" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 35177 } } peers { permanent_uuid: "a34a15c7b1064fd1a9c09c774f7484e4" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37371 } } peers { permanent_uuid: "2364216a2f364a2aa8bdf7a6219778f5" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 38947 } }
I20260806 11:40:39.468537 8404 raft_consensus.cc:374] T 60e2d83e507e453e8890063093fe12d8 P 2364216a2f364a2aa8bdf7a6219778f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:40:39.468538 8402 raft_consensus.cc:374] T 60e2d83e507e453e8890063093fe12d8 P a34a15c7b1064fd1a9c09c774f7484e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:40:39.468600 8404 raft_consensus.cc:729] T 60e2d83e507e453e8890063093fe12d8 P 2364216a2f364a2aa8bdf7a6219778f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2364216a2f364a2aa8bdf7a6219778f5, State: Initialized, Role: FOLLOWER
I20260806 11:40:39.468600 8402 raft_consensus.cc:729] T 60e2d83e507e453e8890063093fe12d8 P a34a15c7b1064fd1a9c09c774f7484e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a34a15c7b1064fd1a9c09c774f7484e4, State: Initialized, Role: FOLLOWER
I20260806 11:40:39.469111 8404 consensus_queue.cc:260] T 60e2d83e507e453e8890063093fe12d8 P 2364216a2f364a2aa8bdf7a6219778f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "045ec27f837840c0a95e4f0431c52ef9" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 35177 } } peers { permanent_uuid: "a34a15c7b1064fd1a9c09c774f7484e4" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37371 } } peers { permanent_uuid: "2364216a2f364a2aa8bdf7a6219778f5" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 38947 } }
I20260806 11:40:39.469118 8402 consensus_queue.cc:260] T 60e2d83e507e453e8890063093fe12d8 P a34a15c7b1064fd1a9c09c774f7484e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "045ec27f837840c0a95e4f0431c52ef9" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 35177 } } peers { permanent_uuid: "a34a15c7b1064fd1a9c09c774f7484e4" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37371 } } peers { permanent_uuid: "2364216a2f364a2aa8bdf7a6219778f5" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 38947 } }
I20260806 11:40:39.469300 8403 tablet_bootstrap.cc:492] T 60e2d83e507e453e8890063093fe12d8 P 045ec27f837840c0a95e4f0431c52ef9: No bootstrap required, opened a new log
I20260806 11:40:39.469548 8403 ts_tablet_manager.cc:1397] T 60e2d83e507e453e8890063093fe12d8 P 045ec27f837840c0a95e4f0431c52ef9: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.000s
I20260806 11:40:39.470072 8103 heartbeater.cc:503] Master 127.21.14.254:33591 was elected leader, sending a full tablet report...
I20260806 11:40:39.470366 8404 ts_tablet_manager.cc:1428] T 60e2d83e507e453e8890063093fe12d8 P 2364216a2f364a2aa8bdf7a6219778f5: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 11:40:39.470369 8402 ts_tablet_manager.cc:1428] T 60e2d83e507e453e8890063093fe12d8 P a34a15c7b1064fd1a9c09c774f7484e4: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 11:40:39.471092 8246 heartbeater.cc:503] Master 127.21.14.254:33591 was elected leader, sending a full tablet report...
I20260806 11:40:39.475649 8403 raft_consensus.cc:348] T 60e2d83e507e453e8890063093fe12d8 P 045ec27f837840c0a95e4f0431c52ef9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "045ec27f837840c0a95e4f0431c52ef9" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 35177 } } peers { permanent_uuid: "a34a15c7b1064fd1a9c09c774f7484e4" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37371 } } peers { permanent_uuid: "2364216a2f364a2aa8bdf7a6219778f5" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 38947 } }
I20260806 11:40:39.476013 8403 raft_consensus.cc:374] T 60e2d83e507e453e8890063093fe12d8 P 045ec27f837840c0a95e4f0431c52ef9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:40:39.476092 8403 raft_consensus.cc:729] T 60e2d83e507e453e8890063093fe12d8 P 045ec27f837840c0a95e4f0431c52ef9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 045ec27f837840c0a95e4f0431c52ef9, State: Initialized, Role: FOLLOWER
I20260806 11:40:39.476616 8403 consensus_queue.cc:260] T 60e2d83e507e453e8890063093fe12d8 P 045ec27f837840c0a95e4f0431c52ef9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "045ec27f837840c0a95e4f0431c52ef9" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 35177 } } peers { permanent_uuid: "a34a15c7b1064fd1a9c09c774f7484e4" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37371 } } peers { permanent_uuid: "2364216a2f364a2aa8bdf7a6219778f5" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 38947 } }
I20260806 11:40:39.477368 8384 heartbeater.cc:503] Master 127.21.14.254:33591 was elected leader, sending a full tablet report...
I20260806 11:40:39.477777 8403 ts_tablet_manager.cc:1428] T 60e2d83e507e453e8890063093fe12d8 P 045ec27f837840c0a95e4f0431c52ef9: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
W20260806 11:40:39.642138 8385 tablet.cc:2369] T 60e2d83e507e453e8890063093fe12d8 P 045ec27f837840c0a95e4f0431c52ef9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 11:40:39.670616 8410 raft_consensus.cc:482] T 60e2d83e507e453e8890063093fe12d8 P 2364216a2f364a2aa8bdf7a6219778f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 11:40:39.671047 8410 raft_consensus.cc:504] T 60e2d83e507e453e8890063093fe12d8 P 2364216a2f364a2aa8bdf7a6219778f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "045ec27f837840c0a95e4f0431c52ef9" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 35177 } } peers { permanent_uuid: "a34a15c7b1064fd1a9c09c774f7484e4" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37371 } } peers { permanent_uuid: "2364216a2f364a2aa8bdf7a6219778f5" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 38947 } }
I20260806 11:40:39.672336 8410 leader_election.cc:288] T 60e2d83e507e453e8890063093fe12d8 P 2364216a2f364a2aa8bdf7a6219778f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 045ec27f837840c0a95e4f0431c52ef9 (127.21.14.195:35177), a34a15c7b1064fd1a9c09c774f7484e4 (127.21.14.194:37371)
I20260806 11:40:39.676723 8397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:39.672765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:56515 (local address 127.21.14.195:35177)
0806 11:40:39.672931 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:39.672937 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:39.673290 (+ 353us) server_negotiation.cc:407] Connection header received
0806 11:40:39.673465 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:39.673469 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:39.673527 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:39.673626 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:40:39.673982 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:39.674508 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:39.675711 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:39.676135 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:39.676272 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:39.676326 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:39.676511 (+ 185us) server_negotiation.cc:299] Negotiation successful
0806 11:40:39.676583 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 11:40:39.676839 8399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:39.672773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:49189 (local address 127.21.14.194:37371)
0806 11:40:39.673056 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:39.673061 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:39.673309 (+ 248us) server_negotiation.cc:407] Connection header received
0806 11:40:39.673808 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:39.673813 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:39.673875 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:39.673963 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:40:39.674257 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:39.674762 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:39.675834 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:39.676238 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:39.676377 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:39.676428 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:39.676632 (+ 204us) server_negotiation.cc:299] Negotiation successful
0806 11:40:39.676707 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182}
I20260806 11:40:39.676914 8412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:39.672651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.194:37371 (local address 127.21.14.193:49189)
0806 11:40:39.673229 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:39.673244 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 11:40:39.673354 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:40:39.673993 (+ 639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:40:39.673996 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:40:39.674015 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:40:39.674138 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:39.674144 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:39.674905 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:39.674908 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:39.675699 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:39.675708 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:39.676384 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:39.676388 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:39.676490 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:39.676513 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 11:40:39.676615 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 11:40:39.676691 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"mutex_wait_us":174,"thread_start_us":113,"threads_started":1}
I20260806 11:40:39.676741 8413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:39.672649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.195:35177 (local address 127.21.14.193:56515)
0806 11:40:39.673196 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:39.673212 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 11:40:39.673355 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:40:39.673657 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:40:39.673660 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:40:39.673680 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:40:39.673867 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:39.673874 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:39.674628 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:39.674633 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:39.675572 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:39.675582 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:39.676270 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:39.676274 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:39.676374 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:39.676403 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 11:40:39.676493 (+ 90us) client_negotiation.cc:240] Negotiation successful
0806 11:40:39.676576 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"spinlock_wait_cycles":7424,"thread_start_us":92,"threads_started":1}
I20260806 11:40:39.677662 8191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60e2d83e507e453e8890063093fe12d8" candidate_uuid: "2364216a2f364a2aa8bdf7a6219778f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a34a15c7b1064fd1a9c09c774f7484e4" is_pre_election: true
I20260806 11:40:39.677716 8336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60e2d83e507e453e8890063093fe12d8" candidate_uuid: "2364216a2f364a2aa8bdf7a6219778f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "045ec27f837840c0a95e4f0431c52ef9" is_pre_election: true
I20260806 11:40:39.678041 8336 raft_consensus.cc:2454] T 60e2d83e507e453e8890063093fe12d8 P 045ec27f837840c0a95e4f0431c52ef9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2364216a2f364a2aa8bdf7a6219778f5 in term 0.
I20260806 11:40:39.678041 8191 raft_consensus.cc:2454] T 60e2d83e507e453e8890063093fe12d8 P a34a15c7b1064fd1a9c09c774f7484e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2364216a2f364a2aa8bdf7a6219778f5 in term 0.
I20260806 11:40:39.678620 7987 leader_election.cc:302] T 60e2d83e507e453e8890063093fe12d8 P 2364216a2f364a2aa8bdf7a6219778f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 045ec27f837840c0a95e4f0431c52ef9, 2364216a2f364a2aa8bdf7a6219778f5, a34a15c7b1064fd1a9c09c774f7484e4; no voters:
W20260806 11:40:39.678704 8247 tablet.cc:2369] T 60e2d83e507e453e8890063093fe12d8 P a34a15c7b1064fd1a9c09c774f7484e4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 11:40:39.678973 8410 raft_consensus.cc:2781] T 60e2d83e507e453e8890063093fe12d8 P 2364216a2f364a2aa8bdf7a6219778f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 11:40:39.679077 8410 raft_consensus.cc:482] T 60e2d83e507e453e8890063093fe12d8 P 2364216a2f364a2aa8bdf7a6219778f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 11:40:39.679131 8410 raft_consensus.cc:3037] T 60e2d83e507e453e8890063093fe12d8 P 2364216a2f364a2aa8bdf7a6219778f5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:40:39.680269 8410 raft_consensus.cc:504] T 60e2d83e507e453e8890063093fe12d8 P 2364216a2f364a2aa8bdf7a6219778f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "045ec27f837840c0a95e4f0431c52ef9" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 35177 } } peers { permanent_uuid: "a34a15c7b1064fd1a9c09c774f7484e4" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37371 } } peers { permanent_uuid: "2364216a2f364a2aa8bdf7a6219778f5" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 38947 } }
I20260806 11:40:39.680716 8410 leader_election.cc:288] T 60e2d83e507e453e8890063093fe12d8 P 2364216a2f364a2aa8bdf7a6219778f5 [CANDIDATE]: Term 1 election: Requested vote from peers 045ec27f837840c0a95e4f0431c52ef9 (127.21.14.195:35177), a34a15c7b1064fd1a9c09c774f7484e4 (127.21.14.194:37371)
I20260806 11:40:39.681151 8336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60e2d83e507e453e8890063093fe12d8" candidate_uuid: "2364216a2f364a2aa8bdf7a6219778f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "045ec27f837840c0a95e4f0431c52ef9"
I20260806 11:40:39.681166 8191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60e2d83e507e453e8890063093fe12d8" candidate_uuid: "2364216a2f364a2aa8bdf7a6219778f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a34a15c7b1064fd1a9c09c774f7484e4"
I20260806 11:40:39.681310 8336 raft_consensus.cc:3037] T 60e2d83e507e453e8890063093fe12d8 P 045ec27f837840c0a95e4f0431c52ef9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:40:39.681320 8191 raft_consensus.cc:3037] T 60e2d83e507e453e8890063093fe12d8 P a34a15c7b1064fd1a9c09c774f7484e4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:40:39.682415 8191 raft_consensus.cc:2454] T 60e2d83e507e453e8890063093fe12d8 P a34a15c7b1064fd1a9c09c774f7484e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2364216a2f364a2aa8bdf7a6219778f5 in term 1.
I20260806 11:40:39.682415 8336 raft_consensus.cc:2454] T 60e2d83e507e453e8890063093fe12d8 P 045ec27f837840c0a95e4f0431c52ef9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2364216a2f364a2aa8bdf7a6219778f5 in term 1.
I20260806 11:40:39.682796 7986 leader_election.cc:302] T 60e2d83e507e453e8890063093fe12d8 P 2364216a2f364a2aa8bdf7a6219778f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2364216a2f364a2aa8bdf7a6219778f5, a34a15c7b1064fd1a9c09c774f7484e4; no voters:
I20260806 11:40:39.683043 8410 raft_consensus.cc:2781] T 60e2d83e507e453e8890063093fe12d8 P 2364216a2f364a2aa8bdf7a6219778f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:40:39.683317 8410 raft_consensus.cc:686] T 60e2d83e507e453e8890063093fe12d8 P 2364216a2f364a2aa8bdf7a6219778f5 [term 1 LEADER]: Becoming Leader. State: Replica: 2364216a2f364a2aa8bdf7a6219778f5, State: Running, Role: LEADER
I20260806 11:40:39.683687 8410 consensus_queue.cc:237] T 60e2d83e507e453e8890063093fe12d8 P 2364216a2f364a2aa8bdf7a6219778f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "045ec27f837840c0a95e4f0431c52ef9" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 35177 } } peers { permanent_uuid: "a34a15c7b1064fd1a9c09c774f7484e4" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37371 } } peers { permanent_uuid: "2364216a2f364a2aa8bdf7a6219778f5" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 38947 } }
I20260806 11:40:39.687414 7907 catalog_manager.cc:5495] T 60e2d83e507e453e8890063093fe12d8 P 2364216a2f364a2aa8bdf7a6219778f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2364216a2f364a2aa8bdf7a6219778f5 (127.21.14.193). New cstate: current_term: 1 leader_uuid: "2364216a2f364a2aa8bdf7a6219778f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "045ec27f837840c0a95e4f0431c52ef9" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 35177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a34a15c7b1064fd1a9c09c774f7484e4" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2364216a2f364a2aa8bdf7a6219778f5" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 38947 } health_report { overall_health: HEALTHY } } }
W20260806 11:40:39.716403 8104 tablet.cc:2369] T 60e2d83e507e453e8890063093fe12d8 P 2364216a2f364a2aa8bdf7a6219778f5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 11:40:39.739856 8401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:39.736002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57440 (local address 127.21.14.193:38947)
0806 11:40:39.736145 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:39.736149 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:39.736162 (+ 13us) server_negotiation.cc:407] Connection header received
0806 11:40:39.736245 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:39.736247 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:39.736304 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:39.736398 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 11:40:39.736698 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:39.737250 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:39.738249 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:39.738494 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:39.738589 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:39.739042 (+ 453us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 11:40:39.739498 (+ 456us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 11:40:39.739602 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:39.739681 (+ 79us) server_negotiation.cc:299] Negotiation successful
0806 11:40:39.739736 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 11:40:39.746086 8336 raft_consensus.cc:1261] T 60e2d83e507e453e8890063093fe12d8 P 045ec27f837840c0a95e4f0431c52ef9 [term 1 FOLLOWER]: Refusing update from remote peer 2364216a2f364a2aa8bdf7a6219778f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 11:40:39.746080 8191 raft_consensus.cc:1261] T 60e2d83e507e453e8890063093fe12d8 P a34a15c7b1064fd1a9c09c774f7484e4 [term 1 FOLLOWER]: Refusing update from remote peer 2364216a2f364a2aa8bdf7a6219778f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 11:40:39.746893 8410 consensus_queue.cc:1035] T 60e2d83e507e453e8890063093fe12d8 P 2364216a2f364a2aa8bdf7a6219778f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34a15c7b1064fd1a9c09c774f7484e4" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 11:40:39.747115 8414 consensus_queue.cc:1035] T 60e2d83e507e453e8890063093fe12d8 P 2364216a2f364a2aa8bdf7a6219778f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "045ec27f837840c0a95e4f0431c52ef9" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 35177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 11:40:39.756011 8419 mvcc.cc:205] Tried to move back new op lower bound from 7315523337192251392 to 7315523336947150848. Current Snapshot: MvccSnapshot[applied={T|T < 7315523337192251392}]
I20260806 11:40:39.757261 8418 mvcc.cc:205] Tried to move back new op lower bound from 7315523337192251392 to 7315523336947150848. Current Snapshot: MvccSnapshot[applied={T|T < 7315523337192251392}]
I20260806 11:40:39.763923 7906 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:56194:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 11:40:39.764132 7906 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:56194:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 11:40:39.766626 7906 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 325b0a599c3e4d5b956553f99520cef3: Sending DeleteTablet for 3 replicas of tablet 60e2d83e507e453e8890063093fe12d8
I20260806 11:40:39.767562 8313 tablet_service.cc:1514] Processing DeleteTablet for tablet 60e2d83e507e453e8890063093fe12d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:40:39 UTC) from {username='slave'} at 127.0.0.1:58766
I20260806 11:40:39.767598 8172 tablet_service.cc:1514] Processing DeleteTablet for tablet 60e2d83e507e453e8890063093fe12d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:40:39 UTC) from {username='slave'} at 127.0.0.1:56860
I20260806 11:40:39.768142 8427 tablet_replica.cc:330] T 60e2d83e507e453e8890063093fe12d8 P a34a15c7b1064fd1a9c09c774f7484e4: stopping tablet replica
I20260806 11:40:39.768394 8032 tablet_service.cc:1514] Processing DeleteTablet for tablet 60e2d83e507e453e8890063093fe12d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:40:39 UTC) from {username='slave'} at 127.0.0.1:57424
I20260806 11:40:39.768503 8427 raft_consensus.cc:2229] T 60e2d83e507e453e8890063093fe12d8 P a34a15c7b1064fd1a9c09c774f7484e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 11:40:39.768662 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 7969
I20260806 11:40:39.770255 8427 raft_consensus.cc:2258] T 60e2d83e507e453e8890063093fe12d8 P a34a15c7b1064fd1a9c09c774f7484e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 11:40:39.771117 8428 tablet_replica.cc:330] T 60e2d83e507e453e8890063093fe12d8 P 045ec27f837840c0a95e4f0431c52ef9: stopping tablet replica
I20260806 11:40:39.771592 8428 raft_consensus.cc:2229] T 60e2d83e507e453e8890063093fe12d8 P 045ec27f837840c0a95e4f0431c52ef9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 11:40:39.771945 8428 raft_consensus.cc:2258] T 60e2d83e507e453e8890063093fe12d8 P 045ec27f837840c0a95e4f0431c52ef9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 11:40:39.772341 8427 ts_tablet_manager.cc:1905] T 60e2d83e507e453e8890063093fe12d8 P a34a15c7b1064fd1a9c09c774f7484e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 11:40:39.773555 8428 ts_tablet_manager.cc:1905] T 60e2d83e507e453e8890063093fe12d8 P 045ec27f837840c0a95e4f0431c52ef9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 11:40:39.775275 8427 ts_tablet_manager.cc:1918] T 60e2d83e507e453e8890063093fe12d8 P a34a15c7b1064fd1a9c09c774f7484e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 11:40:39.775419 8427 log.cc:1196] T 60e2d83e507e453e8890063093fe12d8 P a34a15c7b1064fd1a9c09c774f7484e4: Deleting WAL directory at /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786016169045816-21563-0/minicluster-data/ts-1/wal/wals/60e2d83e507e453e8890063093fe12d8
I20260806 11:40:39.775756 8427 ts_tablet_manager.cc:1939] T 60e2d83e507e453e8890063093fe12d8 P a34a15c7b1064fd1a9c09c774f7484e4: Deleting consensus metadata
I20260806 11:40:39.776546 8428 ts_tablet_manager.cc:1918] T 60e2d83e507e453e8890063093fe12d8 P 045ec27f837840c0a95e4f0431c52ef9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 11:40:39.776654 8428 log.cc:1196] T 60e2d83e507e453e8890063093fe12d8 P 045ec27f837840c0a95e4f0431c52ef9: Deleting WAL directory at /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786016169045816-21563-0/minicluster-data/ts-2/wal/wals/60e2d83e507e453e8890063093fe12d8
I20260806 11:40:39.776968 8428 ts_tablet_manager.cc:1939] T 60e2d83e507e453e8890063093fe12d8 P 045ec27f837840c0a95e4f0431c52ef9: Deleting consensus metadata
I20260806 11:40:39.777441 7891 catalog_manager.cc:4841] TS a34a15c7b1064fd1a9c09c774f7484e4 (127.21.14.194:37371): tablet 60e2d83e507e453e8890063093fe12d8 (table test-table [id=eac309b543af47b49efede8d6ed1e074]) successfully deleted
I20260806 11:40:39.777827 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 8106
W20260806 11:40:39.777855 7889 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.14.193:38947 (error 108)
I20260806 11:40:39.779604 7893 catalog_manager.cc:4841] TS 045ec27f837840c0a95e4f0431c52ef9 (127.21.14.195:35177): tablet 60e2d83e507e453e8890063093fe12d8 (table test-table [id=eac309b543af47b49efede8d6ed1e074]) successfully deleted
I20260806 11:40:39.781129 8400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:39.780713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.193:38947 (local address 127.0.0.1:57448)
0806 11:40:39.780890 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:39.780992 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.193:38947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260806 11:40:39.781325 7889 catalog_manager.cc:4584] TS 2364216a2f364a2aa8bdf7a6219778f5 (127.21.14.193:38947): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 60e2d83e507e453e8890063093fe12d8: Network error: Client connection negotiation failed: client connection to 127.21.14.193:38947: connect: Connection refused (error 111)
I20260806 11:40:39.787276 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 8249
I20260806 11:40:39.795732 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 7873
2026-08-06T11:40:39Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1326 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 11:40:39 dist-test-slave-7c35 krb5kdc[8433](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 11:40:39 dist-test-slave-7c35 krb5kdc[8433](info): set up 2 sockets
Aug 06 11:40:39 dist-test-slave-7c35 krb5kdc[8433](info): commencing operation
krb5kdc: starting...
W20260806 11:40:40.901216 21563 mini_kdc.cc:121] Time spent starting KDC: real 1.081s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Aug 06 11:40:40 dist-test-slave-7c35 krb5kdc[8433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016440, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T11:40:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T11:40:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.14.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab.
Entry for principal kudu/127.21.14.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab.
I20260806 11:40:41.042346 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:33409
--webserver_interface=127.21.14.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab
--principal=kudu/127.21.14.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:42977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.14.254:33409
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:40:41.170586 8447 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:40:41.170964 8447 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:40:41.171080 8447 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:40:41.175387 8447 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 11:40:41.175493 8447 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:40:41.175534 8447 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:40:41.175595 8447 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 11:40:41.175653 8447 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 11:40:41.181680 8447 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:42977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.14.254:33409
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab
--principal=kudu/127.21.14.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:33409
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.14.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8447
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:40:41.183043 8447 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:40:41.184198 8447 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:40:41.194337 8452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:40:41.194357 8455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:40:41.194378 8453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:40:41.194418 8447 server_base.cc:1034] running on GCE node
I20260806 11:40:41.195320 8447 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:40:41.196643 8447 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:40:41.197863 8447 hybrid_clock.cc:648] HybridClock initialized: now 1786016441197853 us; error 37 us; skew 500 ppm
Aug 06 11:40:41 dist-test-slave-7c35 krb5kdc[8433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016441, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:40:41.207546 8447 init.cc:377] Logged in from keytab as kudu/127.21.14.254@KRBTEST.COM (short username kudu)
I20260806 11:40:41.208709 8447 webserver.cc:460] Webserver started at http://127.21.14.254:40423/ using document root <none> and password file <none>
I20260806 11:40:41.209349 8447 fs_manager.cc:359] Metadata directory not provided
I20260806 11:40:41.209458 8447 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:40:41.209720 8447 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:40:41.212227 8447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/minicluster-data/master-0/data/instance:
uuid: "d99bbb1cf2e24605bb7e094484a5c09e"
format_stamp: "Formatted at 2026-08-06 11:40:41 on dist-test-slave-7c35"
server_key: "4c93a36ad0d77660e53dbd3a5cee2a79"
server_key_iv: "bed4bd345baeef370ee23dcb3e120186"
server_key_version: "encryptionkey@0"
I20260806 11:40:41.212893 8447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/minicluster-data/master-0/wal/instance:
uuid: "d99bbb1cf2e24605bb7e094484a5c09e"
format_stamp: "Formatted at 2026-08-06 11:40:41 on dist-test-slave-7c35"
server_key: "4c93a36ad0d77660e53dbd3a5cee2a79"
server_key_iv: "bed4bd345baeef370ee23dcb3e120186"
server_key_version: "encryptionkey@0"
I20260806 11:40:41.217181 8447 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 11:40:41.220146 8463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:41.221590 8447 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 11:40:41.221753 8447 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/minicluster-data/master-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/minicluster-data/master-0/wal
uuid: "d99bbb1cf2e24605bb7e094484a5c09e"
format_stamp: "Formatted at 2026-08-06 11:40:41 on dist-test-slave-7c35"
server_key: "4c93a36ad0d77660e53dbd3a5cee2a79"
server_key_iv: "bed4bd345baeef370ee23dcb3e120186"
server_key_version: "encryptionkey@0"
I20260806 11:40:41.221886 8447 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:40:41.241653 8447 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:40:41.245568 8447 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:40:41.245816 8447 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:40:41.254045 8447 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.254:33409
I20260806 11:40:41.254072 8522 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.254:33409 every 8 connection(s)
I20260806 11:40:41.255155 8447 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
I20260806 11:40:41.258137 8523 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:40:41.261663 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 8447
I20260806 11:40:41.261899 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/minicluster-data/master-0/wal/instance
I20260806 11:40:41.262202 21563 external_mini_cluster.cc:1428] Setting key 66b98940fafd5c4acf17971076c40053
I20260806 11:40:41.265010 8523 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d99bbb1cf2e24605bb7e094484a5c09e: Bootstrap starting.
I20260806 11:40:41.268020 8523 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d99bbb1cf2e24605bb7e094484a5c09e: Neither blocks nor log segments found. Creating new log.
I20260806 11:40:41.269074 8523 log.cc:824] T 00000000000000000000000000000000 P d99bbb1cf2e24605bb7e094484a5c09e: Log is configured to *not* fsync() on all Append() calls
I20260806 11:40:41.271971 8523 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d99bbb1cf2e24605bb7e094484a5c09e: No bootstrap required, opened a new log
Aug 06 11:40:41 dist-test-slave-7c35 krb5kdc[8433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016440, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:40:41.275610 8523 raft_consensus.cc:348] T 00000000000000000000000000000000 P d99bbb1cf2e24605bb7e094484a5c09e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d99bbb1cf2e24605bb7e094484a5c09e" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 33409 } }
I20260806 11:40:41.275913 8523 raft_consensus.cc:374] T 00000000000000000000000000000000 P d99bbb1cf2e24605bb7e094484a5c09e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:40:41.275978 8523 raft_consensus.cc:729] T 00000000000000000000000000000000 P d99bbb1cf2e24605bb7e094484a5c09e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d99bbb1cf2e24605bb7e094484a5c09e, State: Initialized, Role: FOLLOWER
I20260806 11:40:41.276536 8523 consensus_queue.cc:260] T 00000000000000000000000000000000 P d99bbb1cf2e24605bb7e094484a5c09e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d99bbb1cf2e24605bb7e094484a5c09e" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 33409 } }
I20260806 11:40:41.276749 8523 raft_consensus.cc:388] T 00000000000000000000000000000000 P d99bbb1cf2e24605bb7e094484a5c09e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 11:40:41.276825 8523 raft_consensus.cc:482] T 00000000000000000000000000000000 P d99bbb1cf2e24605bb7e094484a5c09e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 11:40:41.276957 8523 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d99bbb1cf2e24605bb7e094484a5c09e [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:40:41.278057 8523 raft_consensus.cc:504] T 00000000000000000000000000000000 P d99bbb1cf2e24605bb7e094484a5c09e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d99bbb1cf2e24605bb7e094484a5c09e" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 33409 } }
I20260806 11:40:41.278475 8523 leader_election.cc:302] T 00000000000000000000000000000000 P d99bbb1cf2e24605bb7e094484a5c09e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d99bbb1cf2e24605bb7e094484a5c09e; no voters:
I20260806 11:40:41.278884 8523 leader_election.cc:288] T 00000000000000000000000000000000 P d99bbb1cf2e24605bb7e094484a5c09e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 11:40:41.278967 8528 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d99bbb1cf2e24605bb7e094484a5c09e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:40:41.279189 8528 raft_consensus.cc:686] T 00000000000000000000000000000000 P d99bbb1cf2e24605bb7e094484a5c09e [term 1 LEADER]: Becoming Leader. State: Replica: d99bbb1cf2e24605bb7e094484a5c09e, State: Running, Role: LEADER
I20260806 11:40:41.279516 8526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:41.263938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48928 (local address 127.21.14.254:33409)
0806 11:40:41.264418 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:41.264435 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:41.264478 (+ 43us) server_negotiation.cc:407] Connection header received
0806 11:40:41.265210 (+ 732us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:41.265244 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:41.265625 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:41.265942 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:40:41.267079 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:41.268233 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:41.269455 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:41.270206 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:41.270342 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:41.273320 (+ 2978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:40:41.273326 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:40:41.273342 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:40:41.273371 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:40:41.275803 (+ 2432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:40:41.276391 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:40:41.276398 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:40:41.276405 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:40:41.276567 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:40:41.276928 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:40:41.276932 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:40:41.276933 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:40:41.277431 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:40:41.277691 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:41.277958 (+ 267us) server_negotiation.cc:299] Negotiation successful
0806 11:40:41.278345 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":149,"threads_started":1}
I20260806 11:40:41.279529 8528 consensus_queue.cc:237] T 00000000000000000000000000000000 P d99bbb1cf2e24605bb7e094484a5c09e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d99bbb1cf2e24605bb7e094484a5c09e" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 33409 } }
I20260806 11:40:41.280340 8523 sys_catalog.cc:564] T 00000000000000000000000000000000 P d99bbb1cf2e24605bb7e094484a5c09e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 11:40:41.281438 8530 sys_catalog.cc:455] T 00000000000000000000000000000000 P d99bbb1cf2e24605bb7e094484a5c09e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d99bbb1cf2e24605bb7e094484a5c09e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d99bbb1cf2e24605bb7e094484a5c09e" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 33409 } } }
I20260806 11:40:41.281582 8530 sys_catalog.cc:458] T 00000000000000000000000000000000 P d99bbb1cf2e24605bb7e094484a5c09e [sys.catalog]: This master's current role is: LEADER
I20260806 11:40:41.281899 8529 sys_catalog.cc:455] T 00000000000000000000000000000000 P d99bbb1cf2e24605bb7e094484a5c09e [sys.catalog]: SysCatalogTable state changed. Reason: New leader d99bbb1cf2e24605bb7e094484a5c09e. Latest consensus state: current_term: 1 leader_uuid: "d99bbb1cf2e24605bb7e094484a5c09e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d99bbb1cf2e24605bb7e094484a5c09e" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 33409 } } }
I20260806 11:40:41.281993 8529 sys_catalog.cc:458] T 00000000000000000000000000000000 P d99bbb1cf2e24605bb7e094484a5c09e [sys.catalog]: This master's current role is: LEADER
I20260806 11:40:41.282269 8538 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 11:40:41.286312 8538 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 11:40:41.292860 8538 catalog_manager.cc:1347] Generated new cluster ID: 75cc197dd88f4ef3ae68e7c2084254ed
I20260806 11:40:41.292955 8538 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 11:40:41.310557 8538 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 11:40:41.311699 8538 catalog_manager.cc:1504] Loading token signing keys...
I20260806 11:40:41.320292 8538 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d99bbb1cf2e24605bb7e094484a5c09e: Generated new TSK 0
I20260806 11:40:41.321118 8538 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.14.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab.
Entry for principal kudu/127.21.14.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab.
I20260806 11:40:41.359582 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.193:0
--local_ip_for_outbound_sockets=127.21.14.193
--webserver_interface=127.21.14.193
--webserver_port=0
--tserver_master_addrs=127.21.14.254:33409
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab
--principal=kudu/127.21.14.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:42977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:40:41.484586 8550 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:40:41.484967 8550 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:40:41.485061 8550 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:40:41.489500 8550 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:40:41.489611 8550 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:40:41.489713 8550 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.193
I20260806 11:40:41.495626 8550 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:42977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab
--principal=kudu/127.21.14.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.14.193
--webserver_port=0
--tserver_master_addrs=127.21.14.254:33409
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8550
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.193
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:40:41.496866 8550 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:40:41.498023 8550 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 11:40:41.508972 8550 server_base.cc:1034] running on GCE node
W20260806 11:40:41.508947 8556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:40:41.508944 8555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:40:41.509168 8558 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:40:41.509719 8550 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:40:41.510479 8550 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:40:41.511765 8550 hybrid_clock.cc:648] HybridClock initialized: now 1786016441511709 us; error 73 us; skew 500 ppm
Aug 06 11:40:41 dist-test-slave-7c35 krb5kdc[8433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016441, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:40:41.515360 8550 init.cc:377] Logged in from keytab as kudu/127.21.14.193@KRBTEST.COM (short username kudu)
I20260806 11:40:41.516388 8550 webserver.cc:460] Webserver started at http://127.21.14.193:41799/ using document root <none> and password file <none>
I20260806 11:40:41.516969 8550 fs_manager.cc:359] Metadata directory not provided
I20260806 11:40:41.517031 8550 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:40:41.517220 8550 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:40:41.519498 8550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/minicluster-data/ts-0/data/instance:
uuid: "6fd89659db2b487c97f96a688454ffee"
format_stamp: "Formatted at 2026-08-06 11:40:41 on dist-test-slave-7c35"
server_key: "79081596344b02dd36f9bf50a60f8c4c"
server_key_iv: "08f21a59d77ea8629bb3f751e88ec4c8"
server_key_version: "encryptionkey@0"
I20260806 11:40:41.520084 8550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/minicluster-data/ts-0/wal/instance:
uuid: "6fd89659db2b487c97f96a688454ffee"
format_stamp: "Formatted at 2026-08-06 11:40:41 on dist-test-slave-7c35"
server_key: "79081596344b02dd36f9bf50a60f8c4c"
server_key_iv: "08f21a59d77ea8629bb3f751e88ec4c8"
server_key_version: "encryptionkey@0"
I20260806 11:40:41.524281 8550 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 11:40:41.527446 8568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:41.529381 8550 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 11:40:41.529558 8550 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/minicluster-data/ts-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/minicluster-data/ts-0/wal
uuid: "6fd89659db2b487c97f96a688454ffee"
format_stamp: "Formatted at 2026-08-06 11:40:41 on dist-test-slave-7c35"
server_key: "79081596344b02dd36f9bf50a60f8c4c"
server_key_iv: "08f21a59d77ea8629bb3f751e88ec4c8"
server_key_version: "encryptionkey@0"
I20260806 11:40:41.529734 8550 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:40:41.560420 8550 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:40:41.564229 8550 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:40:41.564465 8550 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:40:41.565145 8550 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:40:41.566154 8550 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:40:41.566228 8550 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:41.566303 8550 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:40:41.566347 8550 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:41.575790 8550 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.193:42335
I20260806 11:40:41.575810 8685 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.193:42335 every 8 connection(s)
I20260806 11:40:41.576781 8550 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
I20260806 11:40:41.579170 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 8550
I20260806 11:40:41.579331 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/minicluster-data/ts-0/wal/instance
I20260806 11:40:41.579711 21563 external_mini_cluster.cc:1428] Setting key 53223fbc1e6128f71cd3957a8c25a666
Aug 06 11:40:41 dist-test-slave-7c35 krb5kdc[8433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016441, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.193@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.14.194@KRBTEST.COM; defaulting to no policy
I20260806 11:40:41.595036 8526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:41.578901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:45943 (local address 127.21.14.254:33409)
0806 11:40:41.579053 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:41.579058 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:41.580209 (+ 1151us) server_negotiation.cc:407] Connection header received
0806 11:40:41.582360 (+ 2151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:41.582363 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:41.582450 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:41.582533 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:40:41.584462 (+ 1929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:41.585361 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:41.586791 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:41.587270 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:41.587403 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:41.590474 (+ 3071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:40:41.590479 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:40:41.590482 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:40:41.590515 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:40:41.592218 (+ 1703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:40:41.592949 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:40:41.592953 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:40:41.592955 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:40:41.593047 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:40:41.593557 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:40:41.593560 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:40:41.593562 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:40:41.593794 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:40:41.593897 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:41.594621 (+ 724us) server_negotiation.cc:299] Negotiation successful
0806 11:40:41.594759 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.194@KRBTEST.COM" created.
I20260806 11:40:41.596184 8688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:41.579381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:33409 (local address 127.21.14.193:45943)
0806 11:40:41.580004 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:41.580072 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0806 11:40:41.582039 (+ 1967us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:40:41.582817 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:40:41.582831 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:40:41.583496 (+ 665us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:40:41.584272 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:41.584290 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:41.585539 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:41.585542 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:41.586640 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:41.586652 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:41.587461 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:41.587465 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:41.587645 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:41.588377 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:40:41.588395 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:40:41.590269 (+ 1874us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:40:41.592406 (+ 2137us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:40:41.592412 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:40:41.592429 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:40:41.592777 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:40:41.593195 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:40:41.593198 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:40:41.593199 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:40:41.593373 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:40:41.593946 (+ 573us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:40:41.593953 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:40:41.594275 (+ 322us) client_negotiation.cc:769] Sending connection context
0806 11:40:41.594559 (+ 284us) client_negotiation.cc:240] Negotiation successful
0806 11:40:41.594962 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":115,"threads_started":1}
I20260806 11:40:41.597805 8686 heartbeater.cc:347] Connected to a master server at 127.21.14.254:33409
I20260806 11:40:41.598125 8686 heartbeater.cc:464] Registering TS with master...
I20260806 11:40:41.599078 8686 heartbeater.cc:511] Master 127.21.14.254:33409 requested a full tablet report, sending...
I20260806 11:40:41.601064 8482 ts_manager.cc:194] Registered new tserver with Master: 6fd89659db2b487c97f96a688454ffee (127.21.14.193:42335)
I20260806 11:40:41.602368 8482 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.14.193@KRBTEST.COM'} at 127.21.14.193:45943
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab.
Entry for principal kudu/127.21.14.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab.
I20260806 11:40:41.615875 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.194:0
--local_ip_for_outbound_sockets=127.21.14.194
--webserver_interface=127.21.14.194
--webserver_port=0
--tserver_master_addrs=127.21.14.254:33409
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab
--principal=kudu/127.21.14.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:42977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:40:41.744396 8691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:40:41.744748 8691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:40:41.744839 8691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:40:41.749218 8691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:40:41.749329 8691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:40:41.749429 8691 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.194
I20260806 11:40:41.755218 8691 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:42977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab
--principal=kudu/127.21.14.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.14.194
--webserver_port=0
--tserver_master_addrs=127.21.14.254:33409
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8691
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.194
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:40:41.756598 8691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:40:41.757779 8691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:40:41.768446 8697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:40:41.768450 8696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:40:41.768644 8691 server_base.cc:1034] running on GCE node
W20260806 11:40:41.768491 8699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:40:41.769191 8691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:40:41.769903 8691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:40:41.771102 8691 hybrid_clock.cc:648] HybridClock initialized: now 1786016441771059 us; error 54 us; skew 500 ppm
Aug 06 11:40:41 dist-test-slave-7c35 krb5kdc[8433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016441, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:40:41.774137 8691 init.cc:377] Logged in from keytab as kudu/127.21.14.194@KRBTEST.COM (short username kudu)
I20260806 11:40:41.775233 8691 webserver.cc:460] Webserver started at http://127.21.14.194:42681/ using document root <none> and password file <none>
I20260806 11:40:41.775856 8691 fs_manager.cc:359] Metadata directory not provided
I20260806 11:40:41.775962 8691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:40:41.776213 8691 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:40:41.778499 8691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/minicluster-data/ts-1/data/instance:
uuid: "9b167dd5539a4edf8551b99591b519c2"
format_stamp: "Formatted at 2026-08-06 11:40:41 on dist-test-slave-7c35"
server_key: "9bb90619b4f77bec336e52d590d42984"
server_key_iv: "257f4eb62370983e6ae25f515d25d924"
server_key_version: "encryptionkey@0"
I20260806 11:40:41.779150 8691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/minicluster-data/ts-1/wal/instance:
uuid: "9b167dd5539a4edf8551b99591b519c2"
format_stamp: "Formatted at 2026-08-06 11:40:41 on dist-test-slave-7c35"
server_key: "9bb90619b4f77bec336e52d590d42984"
server_key_iv: "257f4eb62370983e6ae25f515d25d924"
server_key_version: "encryptionkey@0"
I20260806 11:40:41.783638 8691 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 11:40:41.786537 8706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:41.788440 8691 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 11:40:41.788626 8691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/minicluster-data/ts-1/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/minicluster-data/ts-1/wal
uuid: "9b167dd5539a4edf8551b99591b519c2"
format_stamp: "Formatted at 2026-08-06 11:40:41 on dist-test-slave-7c35"
server_key: "9bb90619b4f77bec336e52d590d42984"
server_key_iv: "257f4eb62370983e6ae25f515d25d924"
server_key_version: "encryptionkey@0"
I20260806 11:40:41.788774 8691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:40:41.819101 8691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:40:41.823305 8691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:40:41.823571 8691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:40:41.824270 8691 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:40:41.825284 8691 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:40:41.825388 8691 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:41.825460 8691 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:40:41.825519 8691 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:41.836625 8691 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.194:44413
I20260806 11:40:41.836637 8832 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.194:44413 every 8 connection(s)
I20260806 11:40:41.837800 8691 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
I20260806 11:40:41.847905 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 8691
I20260806 11:40:41.848040 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/minicluster-data/ts-1/wal/instance
I20260806 11:40:41.848359 21563 external_mini_cluster.cc:1428] Setting key b1932c339edd51c6194478ffbafe03ae
Aug 06 11:40:41 dist-test-slave-7c35 krb5kdc[8433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016441, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.194@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:40:41.853799 8526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:41.840092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:50523 (local address 127.21.14.254:33409)
0806 11:40:41.840294 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:41.840299 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:41.841198 (+ 899us) server_negotiation.cc:407] Connection header received
0806 11:40:41.842727 (+ 1529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:41.842731 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:41.842796 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:41.842975 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:40:41.844344 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:41.844911 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:41.846230 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:41.846470 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:41.846604 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:41.849731 (+ 3127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:40:41.849740 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:40:41.849743 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:40:41.849769 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:40:41.851450 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:40:41.852079 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:40:41.852083 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:40:41.852085 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:40:41.852175 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:40:41.852574 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:40:41.852577 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:40:41.852579 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:40:41.852790 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:40:41.852905 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:41.853477 (+ 572us) server_negotiation.cc:299] Negotiation successful
0806 11:40:41.853621 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260806 11:40:41.854738 8835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:41.840497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:33409 (local address 127.21.14.194:50523)
0806 11:40:41.841017 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:41.841068 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 11:40:41.842473 (+ 1405us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:40:41.843190 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:40:41.843198 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:40:41.843613 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:40:41.844159 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:41.844175 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:41.845085 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:41.845089 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:41.846071 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:41.846083 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:41.846662 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:41.846667 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:41.846897 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:41.847644 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:40:41.847665 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:40:41.849523 (+ 1858us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:40:41.851624 (+ 2101us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:40:41.851633 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:40:41.851656 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:40:41.851959 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:40:41.852277 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:40:41.852280 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:40:41.852282 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:40:41.852452 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:40:41.852908 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:40:41.852918 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:40:41.853220 (+ 302us) client_negotiation.cc:769] Sending connection context
0806 11:40:41.853435 (+ 215us) client_negotiation.cc:240] Negotiation successful
0806 11:40:41.853833 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":121,"threads_started":1}
I20260806 11:40:41.856105 8833 heartbeater.cc:347] Connected to a master server at 127.21.14.254:33409
I20260806 11:40:41.856391 8833 heartbeater.cc:464] Registering TS with master...
I20260806 11:40:41.856968 8833 heartbeater.cc:511] Master 127.21.14.254:33409 requested a full tablet report, sending...
I20260806 11:40:41.858309 8482 ts_manager.cc:194] Registered new tserver with Master: 9b167dd5539a4edf8551b99591b519c2 (127.21.14.194:44413)
I20260806 11:40:41.859062 8482 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.14.194@KRBTEST.COM'} at 127.21.14.194:50523
WARNING: no policy specified for kudu/127.21.14.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab.
Entry for principal kudu/127.21.14.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab.
I20260806 11:40:41.882246 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.195:0
--local_ip_for_outbound_sockets=127.21.14.195
--webserver_interface=127.21.14.195
--webserver_port=0
--tserver_master_addrs=127.21.14.254:33409
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab
--principal=kudu/127.21.14.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:42977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:40:42.003294 8839 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:40:42.003657 8839 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:40:42.003743 8839 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:40:42.008313 8839 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:40:42.008419 8839 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:40:42.008517 8839 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.195
I20260806 11:40:42.014334 8839 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:42977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab
--principal=kudu/127.21.14.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.14.195
--webserver_port=0
--tserver_master_addrs=127.21.14.254:33409
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8839
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.195
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:40:42.015645 8839 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:40:42.017019 8839 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:40:42.028276 8847 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:40:42.028268 8844 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:40:42.028268 8845 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:40:42.028781 8839 server_base.cc:1034] running on GCE node
I20260806 11:40:42.029265 8839 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:40:42.030035 8839 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:40:42.031229 8839 hybrid_clock.cc:648] HybridClock initialized: now 1786016442031208 us; error 37 us; skew 500 ppm
Aug 06 11:40:42 dist-test-slave-7c35 krb5kdc[8433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016442, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:40:42.034701 8839 init.cc:377] Logged in from keytab as kudu/127.21.14.195@KRBTEST.COM (short username kudu)
I20260806 11:40:42.035768 8839 webserver.cc:460] Webserver started at http://127.21.14.195:46265/ using document root <none> and password file <none>
I20260806 11:40:42.036569 8839 fs_manager.cc:359] Metadata directory not provided
I20260806 11:40:42.036690 8839 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:40:42.036953 8839 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:40:42.039304 8839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/minicluster-data/ts-2/data/instance:
uuid: "5d553e53b2fc42efb444a3b8e1dfc6ad"
format_stamp: "Formatted at 2026-08-06 11:40:42 on dist-test-slave-7c35"
server_key: "9dc153bc6a72f98eedd2536659de8c74"
server_key_iv: "419055ce260274ebfe5a8fc018f39636"
server_key_version: "encryptionkey@0"
I20260806 11:40:42.039948 8839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/minicluster-data/ts-2/wal/instance:
uuid: "5d553e53b2fc42efb444a3b8e1dfc6ad"
format_stamp: "Formatted at 2026-08-06 11:40:42 on dist-test-slave-7c35"
server_key: "9dc153bc6a72f98eedd2536659de8c74"
server_key_iv: "419055ce260274ebfe5a8fc018f39636"
server_key_version: "encryptionkey@0"
I20260806 11:40:42.044432 8839 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 11:40:42.047307 8855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:42.048764 8839 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 11:40:42.048928 8839 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/minicluster-data/ts-2/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/minicluster-data/ts-2/wal
uuid: "5d553e53b2fc42efb444a3b8e1dfc6ad"
format_stamp: "Formatted at 2026-08-06 11:40:42 on dist-test-slave-7c35"
server_key: "9dc153bc6a72f98eedd2536659de8c74"
server_key_iv: "419055ce260274ebfe5a8fc018f39636"
server_key_version: "encryptionkey@0"
I20260806 11:40:42.049067 8839 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:40:42.065673 8839 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:40:42.069501 8839 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:40:42.069836 8839 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:40:42.070580 8839 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:40:42.071574 8839 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:40:42.071659 8839 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:42.071749 8839 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:40:42.071803 8839 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:42.082428 8839 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.195:44687
I20260806 11:40:42.082533 8974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.195:44687 every 8 connection(s)
I20260806 11:40:42.083535 8839 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
I20260806 11:40:42.091589 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 8839
I20260806 11:40:42.091889 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/minicluster-data/ts-2/wal/instance
I20260806 11:40:42.092300 21563 external_mini_cluster.cc:1428] Setting key b7eb79964058d3a4c7f8794c73f4a65e
Aug 06 11:40:42 dist-test-slave-7c35 krb5kdc[8433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016442, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.195@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:40:42.102607 8526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:42.085584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:44101 (local address 127.21.14.254:33409)
0806 11:40:42.085755 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:42.085761 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:42.086740 (+ 979us) server_negotiation.cc:407] Connection header received
0806 11:40:42.088543 (+ 1803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:42.088547 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:42.088604 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:42.088687 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:40:42.090391 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:42.091080 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:42.092699 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:42.093030 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:42.093153 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:42.096760 (+ 3607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:40:42.096764 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:40:42.096767 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:40:42.096801 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:40:42.099240 (+ 2439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:40:42.100037 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:40:42.100040 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:40:42.100041 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:40:42.100135 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:40:42.100885 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:40:42.100888 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:40:42.100889 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:40:42.101265 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:40:42.101377 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:42.102111 (+ 734us) server_negotiation.cc:299] Negotiation successful
0806 11:40:42.102457 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 11:40:42.103935 8977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:42.086058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:33409 (local address 127.21.14.195:44101)
0806 11:40:42.086574 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:42.086622 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 11:40:42.088153 (+ 1531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:40:42.088981 (+ 828us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:40:42.088990 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:40:42.089585 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:40:42.090191 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:42.090211 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:42.091244 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:42.091250 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:42.092461 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:42.092474 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:42.093310 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:42.093314 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:42.093559 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:42.094450 (+ 891us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:40:42.094479 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:40:42.096530 (+ 2051us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:40:42.099450 (+ 2920us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:40:42.099458 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:40:42.099478 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:40:42.099831 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:40:42.100525 (+ 694us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:40:42.100528 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:40:42.100529 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:40:42.100727 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:40:42.101414 (+ 687us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:40:42.101420 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:40:42.101763 (+ 343us) client_negotiation.cc:769] Sending connection context
0806 11:40:42.102048 (+ 285us) client_negotiation.cc:240] Negotiation successful
0806 11:40:42.102601 (+ 553us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":121,"threads_started":1}
I20260806 11:40:42.106182 8975 heartbeater.cc:347] Connected to a master server at 127.21.14.254:33409
I20260806 11:40:42.106571 8975 heartbeater.cc:464] Registering TS with master...
I20260806 11:40:42.107381 8975 heartbeater.cc:511] Master 127.21.14.254:33409 requested a full tablet report, sending...
I20260806 11:40:42.109014 8482 ts_manager.cc:194] Registered new tserver with Master: 5d553e53b2fc42efb444a3b8e1dfc6ad (127.21.14.195:44687)
I20260806 11:40:42.109902 8482 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.14.195@KRBTEST.COM'} at 127.21.14.195:44101
I20260806 11:40:42.122125 21563 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 11:40:42.134325 8526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:42.124729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48932 (local address 127.21.14.254:33409)
0806 11:40:42.124999 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:42.125005 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:42.125165 (+ 160us) server_negotiation.cc:407] Connection header received
0806 11:40:42.125372 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:42.125376 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:42.125428 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:42.125533 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:40:42.126419 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:42.127065 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:42.128014 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:42.128249 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:42.128398 (+ 149us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:42.130454 (+ 2056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:40:42.130458 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:40:42.130460 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:40:42.130492 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:40:42.132088 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:40:42.132750 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:40:42.132754 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:40:42.132756 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:40:42.132846 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:40:42.133269 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:40:42.133272 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:40:42.133274 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:40:42.133518 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:40:42.133684 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:42.134039 (+ 355us) server_negotiation.cc:299] Negotiation successful
0806 11:40:42.134151 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
I20260806 11:40:42.138660 8482 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48932:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 11:40:42.141999 8482 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 11:40:42.159942 8990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:42.153464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.195:44687 (local address 127.0.0.1:57110)
0806 11:40:42.154335 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:42.154349 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 11:40:42.154467 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:40:42.155368 (+ 901us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:40:42.155371 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:40:42.155389 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:40:42.155483 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:42.155489 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:42.156763 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:42.156767 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:42.157888 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:42.157912 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:42.159142 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:42.159147 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:42.159391 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:42.159448 (+ 57us) client_negotiation.cc:769] Sending connection context
0806 11:40:42.159583 (+ 135us) client_negotiation.cc:240] Negotiation successful
0806 11:40:42.159741 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":807,"mutex_wait_us":4,"spinlock_wait_cycles":3456,"thread_start_us":100,"threads_started":1}
I20260806 11:40:42.159950 8993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:42.153917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57110 (local address 127.21.14.195:44687)
0806 11:40:42.154714 (+ 797us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:42.154731 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:42.154753 (+ 22us) server_negotiation.cc:407] Connection header received
0806 11:40:42.154829 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:42.154837 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:42.155070 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:42.155215 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:40:42.155610 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:42.156616 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:42.158046 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:42.158986 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:42.159109 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:42.159292 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:42.159647 (+ 355us) server_negotiation.cc:299] Negotiation successful
0806 11:40:42.159783 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":692,"thread_start_us":134,"threads_started":1}
I20260806 11:40:42.159997 8987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:42.152691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.194:44413 (local address 127.0.0.1:49476)
0806 11:40:42.153272 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:42.153328 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 11:40:42.153462 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:40:42.154419 (+ 957us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:40:42.154425 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:40:42.154466 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:40:42.154633 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:42.154641 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:42.156209 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:42.156216 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:42.158200 (+ 1984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:42.158205 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:42.159538 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:42.159541 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:42.159675 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:42.159700 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 11:40:42.159772 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 11:40:42.159850 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":407,"mutex_wait_us":235,"spinlock_wait_cycles":7552,"thread_start_us":103,"threads_started":1}
I20260806 11:40:42.160485 8989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:42.152885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49476 (local address 127.21.14.194:44413)
0806 11:40:42.153794 (+ 909us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:42.153807 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:42.153831 (+ 24us) server_negotiation.cc:407] Connection header received
0806 11:40:42.153916 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:42.153921 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:42.154173 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:42.154793 (+ 620us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:40:42.155022 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:42.156038 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:42.158349 (+ 2311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:42.159249 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:42.159365 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:42.159481 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:42.160217 (+ 736us) server_negotiation.cc:299] Negotiation successful
0806 11:40:42.160328 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":790,"thread_start_us":151,"threads_started":1}
I20260806 11:40:42.160960 8988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:42.152890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.193:42335 (local address 127.0.0.1:50834)
0806 11:40:42.153400 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:42.153415 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 11:40:42.153524 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:40:42.154672 (+ 1148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:40:42.154676 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:40:42.154692 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:40:42.154799 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:42.154805 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:42.156317 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:42.156320 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:42.158192 (+ 1872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:42.158205 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:42.160512 (+ 2307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:42.160517 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:42.160619 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:42.160646 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 11:40:42.160724 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 11:40:42.160803 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":436,"mutex_wait_us":150,"thread_start_us":123,"threads_started":1}
I20260806 11:40:42.161839 8991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:42.153604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50834 (local address 127.21.14.193:42335)
0806 11:40:42.154039 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:42.154049 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:42.154078 (+ 29us) server_negotiation.cc:407] Connection header received
0806 11:40:42.154154 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:42.154159 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:42.154350 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:42.154509 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:40:42.154993 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:42.156194 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:42.158370 (+ 2176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:42.159480 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:42.159639 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:42.159788 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:42.161550 (+ 1762us) server_negotiation.cc:299] Negotiation successful
0806 11:40:42.161688 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":231,"thread_start_us":111,"threads_started":1}
I20260806 11:40:42.163793 8905 tablet_service.cc:1467] Processing CreateTablet for tablet 6a2219f897394435aafe49a60aa6b2e2 (DEFAULT_TABLE table=test-table [id=67ce4278959044e5a2dc2c854e1c9d7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:40:42.163817 8758 tablet_service.cc:1467] Processing CreateTablet for tablet 6a2219f897394435aafe49a60aa6b2e2 (DEFAULT_TABLE table=test-table [id=67ce4278959044e5a2dc2c854e1c9d7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:40:42.164891 8905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a2219f897394435aafe49a60aa6b2e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:40:42.164927 8758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a2219f897394435aafe49a60aa6b2e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:40:42.164785 8616 tablet_service.cc:1467] Processing CreateTablet for tablet 6a2219f897394435aafe49a60aa6b2e2 (DEFAULT_TABLE table=test-table [id=67ce4278959044e5a2dc2c854e1c9d7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:40:42.165833 8616 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a2219f897394435aafe49a60aa6b2e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:40:42.173240 8997 tablet_bootstrap.cc:492] T 6a2219f897394435aafe49a60aa6b2e2 P 6fd89659db2b487c97f96a688454ffee: Bootstrap starting.
I20260806 11:40:42.173667 8994 tablet_bootstrap.cc:492] T 6a2219f897394435aafe49a60aa6b2e2 P 5d553e53b2fc42efb444a3b8e1dfc6ad: Bootstrap starting.
I20260806 11:40:42.174227 8996 tablet_bootstrap.cc:492] T 6a2219f897394435aafe49a60aa6b2e2 P 9b167dd5539a4edf8551b99591b519c2: Bootstrap starting.
I20260806 11:40:42.175926 8997 tablet_bootstrap.cc:654] T 6a2219f897394435aafe49a60aa6b2e2 P 6fd89659db2b487c97f96a688454ffee: Neither blocks nor log segments found. Creating new log.
I20260806 11:40:42.175926 8994 tablet_bootstrap.cc:654] T 6a2219f897394435aafe49a60aa6b2e2 P 5d553e53b2fc42efb444a3b8e1dfc6ad: Neither blocks nor log segments found. Creating new log.
I20260806 11:40:42.176460 8996 tablet_bootstrap.cc:654] T 6a2219f897394435aafe49a60aa6b2e2 P 9b167dd5539a4edf8551b99591b519c2: Neither blocks nor log segments found. Creating new log.
I20260806 11:40:42.176981 8997 log.cc:824] T 6a2219f897394435aafe49a60aa6b2e2 P 6fd89659db2b487c97f96a688454ffee: Log is configured to *not* fsync() on all Append() calls
I20260806 11:40:42.177307 8994 log.cc:824] T 6a2219f897394435aafe49a60aa6b2e2 P 5d553e53b2fc42efb444a3b8e1dfc6ad: Log is configured to *not* fsync() on all Append() calls
I20260806 11:40:42.177625 8996 log.cc:824] T 6a2219f897394435aafe49a60aa6b2e2 P 9b167dd5539a4edf8551b99591b519c2: Log is configured to *not* fsync() on all Append() calls
I20260806 11:40:42.179486 8997 tablet_bootstrap.cc:492] T 6a2219f897394435aafe49a60aa6b2e2 P 6fd89659db2b487c97f96a688454ffee: No bootstrap required, opened a new log
I20260806 11:40:42.179580 8994 tablet_bootstrap.cc:492] T 6a2219f897394435aafe49a60aa6b2e2 P 5d553e53b2fc42efb444a3b8e1dfc6ad: No bootstrap required, opened a new log
I20260806 11:40:42.179666 8996 tablet_bootstrap.cc:492] T 6a2219f897394435aafe49a60aa6b2e2 P 9b167dd5539a4edf8551b99591b519c2: No bootstrap required, opened a new log
I20260806 11:40:42.179731 8997 ts_tablet_manager.cc:1397] T 6a2219f897394435aafe49a60aa6b2e2 P 6fd89659db2b487c97f96a688454ffee: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 11:40:42.179790 8994 ts_tablet_manager.cc:1397] T 6a2219f897394435aafe49a60aa6b2e2 P 5d553e53b2fc42efb444a3b8e1dfc6ad: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 11:40:42.180013 8996 ts_tablet_manager.cc:1397] T 6a2219f897394435aafe49a60aa6b2e2 P 9b167dd5539a4edf8551b99591b519c2: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 11:40:42.183763 8997 raft_consensus.cc:348] T 6a2219f897394435aafe49a60aa6b2e2 P 6fd89659db2b487c97f96a688454ffee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b167dd5539a4edf8551b99591b519c2" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 44413 } } peers { permanent_uuid: "6fd89659db2b487c97f96a688454ffee" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42335 } } peers { permanent_uuid: "5d553e53b2fc42efb444a3b8e1dfc6ad" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 44687 } }
I20260806 11:40:42.184044 8997 raft_consensus.cc:374] T 6a2219f897394435aafe49a60aa6b2e2 P 6fd89659db2b487c97f96a688454ffee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:40:42.184087 8997 raft_consensus.cc:729] T 6a2219f897394435aafe49a60aa6b2e2 P 6fd89659db2b487c97f96a688454ffee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fd89659db2b487c97f96a688454ffee, State: Initialized, Role: FOLLOWER
I20260806 11:40:42.184204 8994 raft_consensus.cc:348] T 6a2219f897394435aafe49a60aa6b2e2 P 5d553e53b2fc42efb444a3b8e1dfc6ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b167dd5539a4edf8551b99591b519c2" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 44413 } } peers { permanent_uuid: "6fd89659db2b487c97f96a688454ffee" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42335 } } peers { permanent_uuid: "5d553e53b2fc42efb444a3b8e1dfc6ad" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 44687 } }
I20260806 11:40:42.184531 8994 raft_consensus.cc:374] T 6a2219f897394435aafe49a60aa6b2e2 P 5d553e53b2fc42efb444a3b8e1dfc6ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:40:42.184409 8996 raft_consensus.cc:348] T 6a2219f897394435aafe49a60aa6b2e2 P 9b167dd5539a4edf8551b99591b519c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b167dd5539a4edf8551b99591b519c2" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 44413 } } peers { permanent_uuid: "6fd89659db2b487c97f96a688454ffee" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42335 } } peers { permanent_uuid: "5d553e53b2fc42efb444a3b8e1dfc6ad" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 44687 } }
I20260806 11:40:42.184592 8994 raft_consensus.cc:729] T 6a2219f897394435aafe49a60aa6b2e2 P 5d553e53b2fc42efb444a3b8e1dfc6ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d553e53b2fc42efb444a3b8e1dfc6ad, State: Initialized, Role: FOLLOWER
I20260806 11:40:42.184649 8996 raft_consensus.cc:374] T 6a2219f897394435aafe49a60aa6b2e2 P 9b167dd5539a4edf8551b99591b519c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:40:42.184710 8996 raft_consensus.cc:729] T 6a2219f897394435aafe49a60aa6b2e2 P 9b167dd5539a4edf8551b99591b519c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b167dd5539a4edf8551b99591b519c2, State: Initialized, Role: FOLLOWER
I20260806 11:40:42.184595 8997 consensus_queue.cc:260] T 6a2219f897394435aafe49a60aa6b2e2 P 6fd89659db2b487c97f96a688454ffee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b167dd5539a4edf8551b99591b519c2" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 44413 } } peers { permanent_uuid: "6fd89659db2b487c97f96a688454ffee" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42335 } } peers { permanent_uuid: "5d553e53b2fc42efb444a3b8e1dfc6ad" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 44687 } }
I20260806 11:40:42.185096 8994 consensus_queue.cc:260] T 6a2219f897394435aafe49a60aa6b2e2 P 5d553e53b2fc42efb444a3b8e1dfc6ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b167dd5539a4edf8551b99591b519c2" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 44413 } } peers { permanent_uuid: "6fd89659db2b487c97f96a688454ffee" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42335 } } peers { permanent_uuid: "5d553e53b2fc42efb444a3b8e1dfc6ad" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 44687 } }
I20260806 11:40:42.185096 8996 consensus_queue.cc:260] T 6a2219f897394435aafe49a60aa6b2e2 P 9b167dd5539a4edf8551b99591b519c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b167dd5539a4edf8551b99591b519c2" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 44413 } } peers { permanent_uuid: "6fd89659db2b487c97f96a688454ffee" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42335 } } peers { permanent_uuid: "5d553e53b2fc42efb444a3b8e1dfc6ad" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 44687 } }
I20260806 11:40:42.185523 8686 heartbeater.cc:503] Master 127.21.14.254:33409 was elected leader, sending a full tablet report...
I20260806 11:40:42.185874 8997 ts_tablet_manager.cc:1428] T 6a2219f897394435aafe49a60aa6b2e2 P 6fd89659db2b487c97f96a688454ffee: Time spent starting tablet: real 0.006s user 0.002s sys 0.004s
I20260806 11:40:42.186126 8994 ts_tablet_manager.cc:1428] T 6a2219f897394435aafe49a60aa6b2e2 P 5d553e53b2fc42efb444a3b8e1dfc6ad: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 11:40:42.186126 8996 ts_tablet_manager.cc:1428] T 6a2219f897394435aafe49a60aa6b2e2 P 9b167dd5539a4edf8551b99591b519c2: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260806 11:40:42.186590 8975 heartbeater.cc:503] Master 127.21.14.254:33409 was elected leader, sending a full tablet report...
I20260806 11:40:42.187114 8833 heartbeater.cc:503] Master 127.21.14.254:33409 was elected leader, sending a full tablet report...
I20260806 11:40:42.197934 9001 raft_consensus.cc:482] T 6a2219f897394435aafe49a60aa6b2e2 P 6fd89659db2b487c97f96a688454ffee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 11:40:42.198292 9001 raft_consensus.cc:504] T 6a2219f897394435aafe49a60aa6b2e2 P 6fd89659db2b487c97f96a688454ffee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b167dd5539a4edf8551b99591b519c2" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 44413 } } peers { permanent_uuid: "6fd89659db2b487c97f96a688454ffee" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42335 } } peers { permanent_uuid: "5d553e53b2fc42efb444a3b8e1dfc6ad" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 44687 } }
I20260806 11:40:42.199575 9001 leader_election.cc:288] T 6a2219f897394435aafe49a60aa6b2e2 P 6fd89659db2b487c97f96a688454ffee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9b167dd5539a4edf8551b99591b519c2 (127.21.14.194:44413), 5d553e53b2fc42efb444a3b8e1dfc6ad (127.21.14.195:44687)
I20260806 11:40:42.204922 8989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:42.200051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:35633 (local address 127.21.14.194:44413)
0806 11:40:42.200232 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:42.200236 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:42.200584 (+ 348us) server_negotiation.cc:407] Connection header received
0806 11:40:42.200659 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:42.200664 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:42.200744 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:42.200838 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:40:42.201328 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:42.201999 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:42.203521 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:42.204103 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:42.204321 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:42.204376 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:42.204677 (+ 301us) server_negotiation.cc:299] Negotiation successful
0806 11:40:42.204778 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 11:40:42.204977 9004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:42.199936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.194:44413 (local address 127.21.14.193:35633)
0806 11:40:42.200398 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:42.200414 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 11:40:42.200545 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:40:42.200870 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:40:42.200874 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:40:42.200896 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:40:42.201080 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:42.201087 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:42.202252 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:42.202261 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:42.203371 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:42.203382 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:42.204326 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:42.204339 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:42.204457 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:42.204492 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 11:40:42.204599 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 11:40:42.204695 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":379,"mutex_wait_us":1,"thread_start_us":260,"threads_started":1}
I20260806 11:40:42.205211 9005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:42.200570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.195:44687 (local address 127.21.14.193:59399)
0806 11:40:42.200930 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:42.200946 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 11:40:42.201064 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:40:42.201429 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:40:42.201432 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:40:42.201453 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:40:42.201576 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:42.201582 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:42.202650 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:42.202653 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:42.203740 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:42.203751 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:42.204700 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:42.204704 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:42.204813 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:42.204839 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 11:40:42.204940 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 11:40:42.205046 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":110,"threads_started":1}
I20260806 11:40:42.205219 8993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:42.200753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:59399 (local address 127.21.14.195:44687)
0806 11:40:42.200911 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:42.200917 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:42.201037 (+ 120us) server_negotiation.cc:407] Connection header received
0806 11:40:42.201157 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:42.201160 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:42.201302 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:42.201393 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:40:42.201726 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:42.202517 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:42.203968 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:42.204562 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:42.204691 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:42.204748 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:42.204982 (+ 234us) server_negotiation.cc:299] Negotiation successful
0806 11:40:42.205074 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 11:40:42.205899 8781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a2219f897394435aafe49a60aa6b2e2" candidate_uuid: "6fd89659db2b487c97f96a688454ffee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b167dd5539a4edf8551b99591b519c2" is_pre_election: true
I20260806 11:40:42.205899 8925 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a2219f897394435aafe49a60aa6b2e2" candidate_uuid: "6fd89659db2b487c97f96a688454ffee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d553e53b2fc42efb444a3b8e1dfc6ad" is_pre_election: true
I20260806 11:40:42.206272 8925 raft_consensus.cc:2454] T 6a2219f897394435aafe49a60aa6b2e2 P 5d553e53b2fc42efb444a3b8e1dfc6ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fd89659db2b487c97f96a688454ffee in term 0.
I20260806 11:40:42.206272 8781 raft_consensus.cc:2454] T 6a2219f897394435aafe49a60aa6b2e2 P 9b167dd5539a4edf8551b99591b519c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fd89659db2b487c97f96a688454ffee in term 0.
I20260806 11:40:42.206946 8570 leader_election.cc:302] T 6a2219f897394435aafe49a60aa6b2e2 P 6fd89659db2b487c97f96a688454ffee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fd89659db2b487c97f96a688454ffee, 9b167dd5539a4edf8551b99591b519c2; no voters:
I20260806 11:40:42.207311 9001 raft_consensus.cc:2781] T 6a2219f897394435aafe49a60aa6b2e2 P 6fd89659db2b487c97f96a688454ffee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 11:40:42.207424 9001 raft_consensus.cc:482] T 6a2219f897394435aafe49a60aa6b2e2 P 6fd89659db2b487c97f96a688454ffee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 11:40:42.207475 9001 raft_consensus.cc:3037] T 6a2219f897394435aafe49a60aa6b2e2 P 6fd89659db2b487c97f96a688454ffee [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:40:42.208796 9001 raft_consensus.cc:504] T 6a2219f897394435aafe49a60aa6b2e2 P 6fd89659db2b487c97f96a688454ffee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b167dd5539a4edf8551b99591b519c2" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 44413 } } peers { permanent_uuid: "6fd89659db2b487c97f96a688454ffee" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42335 } } peers { permanent_uuid: "5d553e53b2fc42efb444a3b8e1dfc6ad" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 44687 } }
I20260806 11:40:42.209271 9001 leader_election.cc:288] T 6a2219f897394435aafe49a60aa6b2e2 P 6fd89659db2b487c97f96a688454ffee [CANDIDATE]: Term 1 election: Requested vote from peers 9b167dd5539a4edf8551b99591b519c2 (127.21.14.194:44413), 5d553e53b2fc42efb444a3b8e1dfc6ad (127.21.14.195:44687)
I20260806 11:40:42.209753 8925 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a2219f897394435aafe49a60aa6b2e2" candidate_uuid: "6fd89659db2b487c97f96a688454ffee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d553e53b2fc42efb444a3b8e1dfc6ad"
I20260806 11:40:42.209769 8781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a2219f897394435aafe49a60aa6b2e2" candidate_uuid: "6fd89659db2b487c97f96a688454ffee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b167dd5539a4edf8551b99591b519c2"
I20260806 11:40:42.209935 8925 raft_consensus.cc:3037] T 6a2219f897394435aafe49a60aa6b2e2 P 5d553e53b2fc42efb444a3b8e1dfc6ad [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:40:42.209944 8781 raft_consensus.cc:3037] T 6a2219f897394435aafe49a60aa6b2e2 P 9b167dd5539a4edf8551b99591b519c2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:40:42.211266 8781 raft_consensus.cc:2454] T 6a2219f897394435aafe49a60aa6b2e2 P 9b167dd5539a4edf8551b99591b519c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fd89659db2b487c97f96a688454ffee in term 1.
I20260806 11:40:42.211268 8925 raft_consensus.cc:2454] T 6a2219f897394435aafe49a60aa6b2e2 P 5d553e53b2fc42efb444a3b8e1dfc6ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fd89659db2b487c97f96a688454ffee in term 1.
I20260806 11:40:42.211670 8570 leader_election.cc:302] T 6a2219f897394435aafe49a60aa6b2e2 P 6fd89659db2b487c97f96a688454ffee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d553e53b2fc42efb444a3b8e1dfc6ad, 6fd89659db2b487c97f96a688454ffee; no voters:
I20260806 11:40:42.211901 9001 raft_consensus.cc:2781] T 6a2219f897394435aafe49a60aa6b2e2 P 6fd89659db2b487c97f96a688454ffee [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:40:42.212178 9001 raft_consensus.cc:686] T 6a2219f897394435aafe49a60aa6b2e2 P 6fd89659db2b487c97f96a688454ffee [term 1 LEADER]: Becoming Leader. State: Replica: 6fd89659db2b487c97f96a688454ffee, State: Running, Role: LEADER
I20260806 11:40:42.212592 9001 consensus_queue.cc:237] T 6a2219f897394435aafe49a60aa6b2e2 P 6fd89659db2b487c97f96a688454ffee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b167dd5539a4edf8551b99591b519c2" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 44413 } } peers { permanent_uuid: "6fd89659db2b487c97f96a688454ffee" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42335 } } peers { permanent_uuid: "5d553e53b2fc42efb444a3b8e1dfc6ad" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 44687 } }
I20260806 11:40:42.216502 8480 catalog_manager.cc:5495] T 6a2219f897394435aafe49a60aa6b2e2 P 6fd89659db2b487c97f96a688454ffee reported cstate change: term changed from 0 to 1, leader changed from <none> to 6fd89659db2b487c97f96a688454ffee (127.21.14.193). New cstate: current_term: 1 leader_uuid: "6fd89659db2b487c97f96a688454ffee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b167dd5539a4edf8551b99591b519c2" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 44413 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fd89659db2b487c97f96a688454ffee" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 42335 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5d553e53b2fc42efb444a3b8e1dfc6ad" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 44687 } health_report { overall_health: UNKNOWN } } }
I20260806 11:40:42.230019 8991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:42.226142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50838 (local address 127.21.14.193:42335)
0806 11:40:42.226272 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:42.226276 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:42.226291 (+ 15us) server_negotiation.cc:407] Connection header received
0806 11:40:42.226447 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:42.226449 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:42.226505 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:42.226600 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 11:40:42.226878 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:42.227400 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:42.228450 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:42.228682 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:42.228765 (+ 83us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:42.229123 (+ 358us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 11:40:42.229548 (+ 425us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 11:40:42.229662 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:42.229787 (+ 125us) server_negotiation.cc:299] Negotiation successful
0806 11:40:42.229825 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 11:40:42.236896 8781 raft_consensus.cc:1261] T 6a2219f897394435aafe49a60aa6b2e2 P 9b167dd5539a4edf8551b99591b519c2 [term 1 FOLLOWER]: Refusing update from remote peer 6fd89659db2b487c97f96a688454ffee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 11:40:42.236902 8925 raft_consensus.cc:1261] T 6a2219f897394435aafe49a60aa6b2e2 P 5d553e53b2fc42efb444a3b8e1dfc6ad [term 1 FOLLOWER]: Refusing update from remote peer 6fd89659db2b487c97f96a688454ffee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 11:40:42.237691 9001 consensus_queue.cc:1035] T 6a2219f897394435aafe49a60aa6b2e2 P 6fd89659db2b487c97f96a688454ffee [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b167dd5539a4edf8551b99591b519c2" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 44413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 11:40:42.237952 9006 consensus_queue.cc:1035] T 6a2219f897394435aafe49a60aa6b2e2 P 6fd89659db2b487c97f96a688454ffee [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d553e53b2fc42efb444a3b8e1dfc6ad" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 44687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 11:40:42.246433 9010 mvcc.cc:205] Tried to move back new op lower bound from 7315523347394002944 to 7315523347306131456. Current Snapshot: MvccSnapshot[applied={T|T < 7315523347394002944}]
I20260806 11:40:42.247272 9011 mvcc.cc:205] Tried to move back new op lower bound from 7315523347394002944 to 7315523347306131456. Current Snapshot: MvccSnapshot[applied={T|T < 7315523347394002944}]
I20260806 11:40:42.256487 8480 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48932:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 11:40:42.256669 8480 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48932:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 11:40:42.259513 8480 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d99bbb1cf2e24605bb7e094484a5c09e: Sending DeleteTablet for 3 replicas of tablet 6a2219f897394435aafe49a60aa6b2e2
I20260806 11:40:42.260650 8905 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a2219f897394435aafe49a60aa6b2e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:40:42 UTC) from {username='kudu', principal='kudu/127.21.14.254@KRBTEST.COM'} at 127.0.0.1:57110
I20260806 11:40:42.260658 8758 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a2219f897394435aafe49a60aa6b2e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:40:42 UTC) from {username='kudu', principal='kudu/127.21.14.254@KRBTEST.COM'} at 127.0.0.1:49476
I20260806 11:40:42.261524 8616 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a2219f897394435aafe49a60aa6b2e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:40:42 UTC) from {username='kudu', principal='kudu/127.21.14.254@KRBTEST.COM'} at 127.0.0.1:50834
I20260806 11:40:42.262279 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 8550
I20260806 11:40:42.265481 9020 tablet_replica.cc:330] T 6a2219f897394435aafe49a60aa6b2e2 P 9b167dd5539a4edf8551b99591b519c2: stopping tablet replica
I20260806 11:40:42.265923 9020 raft_consensus.cc:2229] T 6a2219f897394435aafe49a60aa6b2e2 P 9b167dd5539a4edf8551b99591b519c2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 11:40:42.266340 9020 raft_consensus.cc:2258] T 6a2219f897394435aafe49a60aa6b2e2 P 9b167dd5539a4edf8551b99591b519c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 11:40:42.262162 9019 tablet_replica.cc:330] T 6a2219f897394435aafe49a60aa6b2e2 P 5d553e53b2fc42efb444a3b8e1dfc6ad: stopping tablet replica
I20260806 11:40:42.267004 9019 raft_consensus.cc:2229] T 6a2219f897394435aafe49a60aa6b2e2 P 5d553e53b2fc42efb444a3b8e1dfc6ad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 11:40:42.267366 9019 raft_consensus.cc:2258] T 6a2219f897394435aafe49a60aa6b2e2 P 5d553e53b2fc42efb444a3b8e1dfc6ad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 11:40:42.268635 9020 ts_tablet_manager.cc:1905] T 6a2219f897394435aafe49a60aa6b2e2 P 9b167dd5539a4edf8551b99591b519c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 11:40:42.268925 9019 ts_tablet_manager.cc:1905] T 6a2219f897394435aafe49a60aa6b2e2 P 5d553e53b2fc42efb444a3b8e1dfc6ad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 11:40:42.271754 9020 ts_tablet_manager.cc:1918] T 6a2219f897394435aafe49a60aa6b2e2 P 9b167dd5539a4edf8551b99591b519c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 11:40:42.271951 9020 log.cc:1196] T 6a2219f897394435aafe49a60aa6b2e2 P 9b167dd5539a4edf8551b99591b519c2: Deleting WAL directory at /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/minicluster-data/ts-1/wal/wals/6a2219f897394435aafe49a60aa6b2e2
I20260806 11:40:42.272382 9020 ts_tablet_manager.cc:1939] T 6a2219f897394435aafe49a60aa6b2e2 P 9b167dd5539a4edf8551b99591b519c2: Deleting consensus metadata
I20260806 11:40:42.272756 9019 ts_tablet_manager.cc:1918] T 6a2219f897394435aafe49a60aa6b2e2 P 5d553e53b2fc42efb444a3b8e1dfc6ad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 11:40:42.272887 9019 log.cc:1196] T 6a2219f897394435aafe49a60aa6b2e2 P 5d553e53b2fc42efb444a3b8e1dfc6ad: Deleting WAL directory at /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786016169045816-21563-0/minicluster-data/ts-2/wal/wals/6a2219f897394435aafe49a60aa6b2e2
I20260806 11:40:42.273273 9019 ts_tablet_manager.cc:1939] T 6a2219f897394435aafe49a60aa6b2e2 P 5d553e53b2fc42efb444a3b8e1dfc6ad: Deleting consensus metadata
I20260806 11:40:42.274005 8465 catalog_manager.cc:4841] TS 9b167dd5539a4edf8551b99591b519c2 (127.21.14.194:44413): tablet 6a2219f897394435aafe49a60aa6b2e2 (table test-table [id=67ce4278959044e5a2dc2c854e1c9d7f]) successfully deleted
I20260806 11:40:42.274636 8465 catalog_manager.cc:4841] TS 5d553e53b2fc42efb444a3b8e1dfc6ad (127.21.14.195:44687): tablet 6a2219f897394435aafe49a60aa6b2e2 (table test-table [id=67ce4278959044e5a2dc2c854e1c9d7f]) successfully deleted
W20260806 11:40:42.275416 8464 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.14.193:42335 (error 108)
I20260806 11:40:42.275720 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 8691
I20260806 11:40:42.277541 8988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:42.277164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.193:42335 (local address 127.0.0.1:50844)
0806 11:40:42.277345 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:42.277432 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.193:42335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260806 11:40:42.277765 8464 catalog_manager.cc:4584] TS 6fd89659db2b487c97f96a688454ffee (127.21.14.193:42335): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6a2219f897394435aafe49a60aa6b2e2: Network error: Client connection negotiation failed: client connection to 127.21.14.193:42335: connect: Connection refused (error 111)
I20260806 11:40:42.284471 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 8839
I20260806 11:40:42.292167 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 8447
2026-08-06T11:40:42Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2496 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-06T11:40:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T11:40:42Z Disabled control of system clock
I20260806 11:40:42.327029 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786016169045816-21563-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786016169045816-21563-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786016169045816-21563-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:34097
--webserver_interface=127.21.14.254
--webserver_port=0
--builtin_ntp_servers=127.21.14.212:40717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.14.254:34097
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 11:40:42.445786 9026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:40:42.446101 9026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:40:42.446139 9026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:40:42.450235 9026 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 11:40:42.450309 9026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:40:42.450325 9026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:40:42.450341 9026 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 11:40:42.450357 9026 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 11:40:42.455435 9026 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:40717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786016169045816-21563-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786016169045816-21563-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.14.254:34097
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:34097
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.14.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9026
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786016169045816-21563-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:40:42.456607 9026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:40:42.458065 9026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:40:42.467689 9032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:40:42.467689 9031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:40:42.467784 9034 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:40:42.467739 9026 server_base.cc:1034] running on GCE node
I20260806 11:40:42.468636 9026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:40:42.469830 9026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:40:42.471055 9026 hybrid_clock.cc:648] HybridClock initialized: now 1786016442471012 us; error 55 us; skew 500 ppm
I20260806 11:40:42.473183 9026 webserver.cc:460] Webserver started at http://127.21.14.254:35125/ using document root <none> and password file <none>
I20260806 11:40:42.473832 9026 fs_manager.cc:359] Metadata directory not provided
I20260806 11:40:42.473941 9026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:40:42.474198 9026 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:40:42.476487 9026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786016169045816-21563-0/minicluster-data/master-0/data/instance:
uuid: "396555bb61944261a44c0e82a7827248"
format_stamp: "Formatted at 2026-08-06 11:40:42 on dist-test-slave-7c35"
server_key: "81faf7051bc96985659f2eebd4842a42"
server_key_iv: "235f329c99e712bab64651dd47dd2609"
server_key_version: "encryptionkey@0"
I20260806 11:40:42.477128 9026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786016169045816-21563-0/minicluster-data/master-0/wal/instance:
uuid: "396555bb61944261a44c0e82a7827248"
format_stamp: "Formatted at 2026-08-06 11:40:42 on dist-test-slave-7c35"
server_key: "81faf7051bc96985659f2eebd4842a42"
server_key_iv: "235f329c99e712bab64651dd47dd2609"
server_key_version: "encryptionkey@0"
I20260806 11:40:42.481441 9026 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 11:40:42.484494 9040 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:42.486045 9026 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 11:40:42.486217 9026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786016169045816-21563-0/minicluster-data/master-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786016169045816-21563-0/minicluster-data/master-0/wal
uuid: "396555bb61944261a44c0e82a7827248"
format_stamp: "Formatted at 2026-08-06 11:40:42 on dist-test-slave-7c35"
server_key: "81faf7051bc96985659f2eebd4842a42"
server_key_iv: "235f329c99e712bab64651dd47dd2609"
server_key_version: "encryptionkey@0"
I20260806 11:40:42.486356 9026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786016169045816-21563-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786016169045816-21563-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786016169045816-21563-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:40:42.510898 9026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:40:42.511737 9026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:40:42.511979 9026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:40:42.520848 9026 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.254:34097
I20260806 11:40:42.520855 9097 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.254:34097 every 8 connection(s)
I20260806 11:40:42.521961 9026 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
I20260806 11:40:42.524303 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 9026
I20260806 11:40:42.524451 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786016169045816-21563-0/minicluster-data/master-0/wal/instance
I20260806 11:40:42.524749 21563 external_mini_cluster.cc:1428] Setting key abd0dd2f31e343af4fb504c1feae0068
I20260806 11:40:42.525739 9098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:40:42.532850 9098 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 396555bb61944261a44c0e82a7827248: Bootstrap starting.
I20260806 11:40:42.534662 9101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:42.526249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48672 (local address 127.21.14.254:34097)
0806 11:40:42.526989 (+ 740us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:42.527006 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:42.527046 (+ 40us) server_negotiation.cc:407] Connection header received
0806 11:40:42.527746 (+ 700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:42.527765 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:42.528183 (+ 418us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:42.528482 (+ 299us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:40:42.528859 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:42.529847 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:42.531119 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:42.531638 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:42.531760 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:42.532714 (+ 954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:40:42.532718 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:40:42.532732 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 11:40:42.533220 (+ 488us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:40:42.533283 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 11:40:42.533305 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:40:42.533385 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:42.533561 (+ 176us) server_negotiation.cc:299] Negotiation successful
0806 11:40:42.533874 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":161,"threads_started":1}
I20260806 11:40:42.535640 9098 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 396555bb61944261a44c0e82a7827248: Neither blocks nor log segments found. Creating new log.
I20260806 11:40:42.536424 9098 log.cc:824] T 00000000000000000000000000000000 P 396555bb61944261a44c0e82a7827248: Log is configured to *not* fsync() on all Append() calls
I20260806 11:40:42.538501 9098 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 396555bb61944261a44c0e82a7827248: No bootstrap required, opened a new log
I20260806 11:40:42.542308 9098 raft_consensus.cc:348] T 00000000000000000000000000000000 P 396555bb61944261a44c0e82a7827248 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "396555bb61944261a44c0e82a7827248" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 34097 } }
I20260806 11:40:42.542588 9098 raft_consensus.cc:374] T 00000000000000000000000000000000 P 396555bb61944261a44c0e82a7827248 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:40:42.542657 9098 raft_consensus.cc:729] T 00000000000000000000000000000000 P 396555bb61944261a44c0e82a7827248 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 396555bb61944261a44c0e82a7827248, State: Initialized, Role: FOLLOWER
I20260806 11:40:42.543227 9098 consensus_queue.cc:260] T 00000000000000000000000000000000 P 396555bb61944261a44c0e82a7827248 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "396555bb61944261a44c0e82a7827248" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 34097 } }
I20260806 11:40:42.543403 9098 raft_consensus.cc:388] T 00000000000000000000000000000000 P 396555bb61944261a44c0e82a7827248 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 11:40:42.543531 9098 raft_consensus.cc:482] T 00000000000000000000000000000000 P 396555bb61944261a44c0e82a7827248 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 11:40:42.543663 9098 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 396555bb61944261a44c0e82a7827248 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:40:42.544754 9098 raft_consensus.cc:504] T 00000000000000000000000000000000 P 396555bb61944261a44c0e82a7827248 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "396555bb61944261a44c0e82a7827248" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 34097 } }
I20260806 11:40:42.545159 9098 leader_election.cc:302] T 00000000000000000000000000000000 P 396555bb61944261a44c0e82a7827248 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 396555bb61944261a44c0e82a7827248; no voters:
I20260806 11:40:42.545531 9098 leader_election.cc:288] T 00000000000000000000000000000000 P 396555bb61944261a44c0e82a7827248 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 11:40:42.545805 9103 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 396555bb61944261a44c0e82a7827248 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:40:42.546083 9103 raft_consensus.cc:686] T 00000000000000000000000000000000 P 396555bb61944261a44c0e82a7827248 [term 1 LEADER]: Becoming Leader. State: Replica: 396555bb61944261a44c0e82a7827248, State: Running, Role: LEADER
I20260806 11:40:42.546433 9103 consensus_queue.cc:237] T 00000000000000000000000000000000 P 396555bb61944261a44c0e82a7827248 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "396555bb61944261a44c0e82a7827248" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 34097 } }
I20260806 11:40:42.547098 9098 sys_catalog.cc:564] T 00000000000000000000000000000000 P 396555bb61944261a44c0e82a7827248 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 11:40:42.548449 9104 sys_catalog.cc:455] T 00000000000000000000000000000000 P 396555bb61944261a44c0e82a7827248 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "396555bb61944261a44c0e82a7827248" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "396555bb61944261a44c0e82a7827248" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 34097 } } }
I20260806 11:40:42.548465 9105 sys_catalog.cc:455] T 00000000000000000000000000000000 P 396555bb61944261a44c0e82a7827248 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 396555bb61944261a44c0e82a7827248. Latest consensus state: current_term: 1 leader_uuid: "396555bb61944261a44c0e82a7827248" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "396555bb61944261a44c0e82a7827248" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 34097 } } }
I20260806 11:40:42.548604 9105 sys_catalog.cc:458] T 00000000000000000000000000000000 P 396555bb61944261a44c0e82a7827248 [sys.catalog]: This master's current role is: LEADER
I20260806 11:40:42.548604 9104 sys_catalog.cc:458] T 00000000000000000000000000000000 P 396555bb61944261a44c0e82a7827248 [sys.catalog]: This master's current role is: LEADER
I20260806 11:40:42.549187 9112 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 11:40:42.552663 9112 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 11:40:42.558920 9112 catalog_manager.cc:1347] Generated new cluster ID: 3bb59353be0947aea87bf1533fa65850
I20260806 11:40:42.559010 9112 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 11:40:42.566656 9112 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 11:40:42.567750 9112 catalog_manager.cc:1504] Loading token signing keys...
I20260806 11:40:42.575243 9112 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 396555bb61944261a44c0e82a7827248: Generated new TSK 0
I20260806 11:40:42.576035 9112 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 11:40:42.580240 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786016169045816-21563-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786016169045816-21563-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786016169045816-21563-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.193:0
--local_ip_for_outbound_sockets=127.21.14.193
--webserver_interface=127.21.14.193
--webserver_port=0
--tserver_master_addrs=127.21.14.254:34097
--builtin_ntp_servers=127.21.14.212:40717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 11:40:42.718075 9122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:40:42.718438 9122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:40:42.718484 9122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:40:42.722774 9122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:40:42.722872 9122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:40:42.722962 9122 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.193
I20260806 11:40:42.728091 9122 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:40717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786016169045816-21563-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786016169045816-21563-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.14.193
--webserver_port=0
--tserver_master_addrs=127.21.14.254:34097
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9122
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.193
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786016169045816-21563-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:40:42.729301 9122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:40:42.730805 9122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:40:42.741276 9128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:40:42.741454 9122 server_base.cc:1034] running on GCE node
W20260806 11:40:42.741359 9131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:40:42.741765 9129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:40:42.742285 9122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:40:42.743058 9122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:40:42.744243 9122 hybrid_clock.cc:648] HybridClock initialized: now 1786016442744213 us; error 46 us; skew 500 ppm
I20260806 11:40:42.746464 9122 webserver.cc:460] Webserver started at http://127.21.14.193:40581/ using document root <none> and password file <none>
I20260806 11:40:42.747196 9122 fs_manager.cc:359] Metadata directory not provided
I20260806 11:40:42.747267 9122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:40:42.747524 9122 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:40:42.749998 9122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786016169045816-21563-0/minicluster-data/ts-0/data/instance:
uuid: "498c59dba5c44ceb92843987b108f5cb"
format_stamp: "Formatted at 2026-08-06 11:40:42 on dist-test-slave-7c35"
server_key: "68a7ef3a1929ca239ffe903b9237f8de"
server_key_iv: "920bb647d2bc21ffd9a3bf921aae8656"
server_key_version: "encryptionkey@0"
I20260806 11:40:42.750700 9122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786016169045816-21563-0/minicluster-data/ts-0/wal/instance:
uuid: "498c59dba5c44ceb92843987b108f5cb"
format_stamp: "Formatted at 2026-08-06 11:40:42 on dist-test-slave-7c35"
server_key: "68a7ef3a1929ca239ffe903b9237f8de"
server_key_iv: "920bb647d2bc21ffd9a3bf921aae8656"
server_key_version: "encryptionkey@0"
I20260806 11:40:42.754846 9122 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 11:40:42.757755 9137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:42.759302 9122 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 11:40:42.759464 9122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786016169045816-21563-0/minicluster-data/ts-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786016169045816-21563-0/minicluster-data/ts-0/wal
uuid: "498c59dba5c44ceb92843987b108f5cb"
format_stamp: "Formatted at 2026-08-06 11:40:42 on dist-test-slave-7c35"
server_key: "68a7ef3a1929ca239ffe903b9237f8de"
server_key_iv: "920bb647d2bc21ffd9a3bf921aae8656"
server_key_version: "encryptionkey@0"
I20260806 11:40:42.759629 9122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786016169045816-21563-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786016169045816-21563-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786016169045816-21563-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:40:42.790685 9122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:40:42.791590 9122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:40:42.791812 9122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:40:42.792526 9122 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:40:42.793649 9122 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:40:42.793725 9122 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:42.793802 9122 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:40:42.793844 9122 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:42.803741 9122 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.193:43465
I20260806 11:40:42.803799 9262 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.193:43465 every 8 connection(s)
I20260806 11:40:42.804755 9122 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
I20260806 11:40:42.810478 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 9122
I20260806 11:40:42.810638 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786016169045816-21563-0/minicluster-data/ts-0/wal/instance
I20260806 11:40:42.811014 21563 external_mini_cluster.cc:1428] Setting key 428dc5103303e009b5d4ba11b81dd2f4
I20260806 11:40:42.813104 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786016169045816-21563-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786016169045816-21563-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786016169045816-21563-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.194:0
--local_ip_for_outbound_sockets=127.21.14.194
--webserver_interface=127.21.14.194
--webserver_port=0
--tserver_master_addrs=127.21.14.254:34097
--builtin_ntp_servers=127.21.14.212:40717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 11:40:42.816041 9101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:42.806916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:40961 (local address 127.21.14.254:34097)
0806 11:40:42.807065 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:42.807068 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:42.808322 (+ 1254us) server_negotiation.cc:407] Connection header received
0806 11:40:42.810106 (+ 1784us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:42.810109 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:42.810184 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:42.810272 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:40:42.811564 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:42.812273 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:42.813641 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:42.813874 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:42.813997 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:42.815478 (+ 1481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:40:42.815481 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:40:42.815483 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 11:40:42.815498 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:40:42.815523 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 11:40:42.815532 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:40:42.815607 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:42.815830 (+ 223us) server_negotiation.cc:299] Negotiation successful
0806 11:40:42.815916 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 11:40:42.817365 9266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:42.807366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:34097 (local address 127.21.14.193:40961)
0806 11:40:42.808159 (+ 793us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:42.808206 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 11:40:42.809775 (+ 1569us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:40:42.810484 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:40:42.810499 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:40:42.810647 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:40:42.811399 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:42.811416 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:42.812435 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:42.812439 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:42.813505 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:42.813516 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:42.814059 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:42.814064 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:42.814259 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:42.815075 (+ 816us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 11:40:42.815079 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:40:42.815158 (+ 79us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 11:40:42.815265 (+ 107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:40:42.815638 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:40:42.815651 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:40:42.815658 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 11:40:42.815801 (+ 143us) client_negotiation.cc:240] Negotiation successful
0806 11:40:42.816187 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":141,"threads_started":1}
I20260806 11:40:42.819089 9263 heartbeater.cc:347] Connected to a master server at 127.21.14.254:34097
I20260806 11:40:42.819408 9263 heartbeater.cc:464] Registering TS with master...
I20260806 11:40:42.820220 9263 heartbeater.cc:511] Master 127.21.14.254:34097 requested a full tablet report, sending...
I20260806 11:40:42.822134 9051 ts_manager.cc:194] Registered new tserver with Master: 498c59dba5c44ceb92843987b108f5cb (127.21.14.193:43465)
I20260806 11:40:42.823454 9051 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.14.193:40961
W20260806 11:40:42.937500 9267 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:40:42.937757 9267 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:40:42.937800 9267 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:40:42.941995 9267 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:40:42.942065 9267 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:40:42.942123 9267 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.194
I20260806 11:40:42.947304 9267 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:40717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786016169045816-21563-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786016169045816-21563-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.14.194
--webserver_port=0
--tserver_master_addrs=127.21.14.254:34097
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9267
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.194
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786016169045816-21563-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:40:42.948644 9267 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:40:42.950228 9267 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:40:42.960448 9272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:40:42.960526 9273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:40:42.960448 9275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:40:42.960569 9267 server_base.cc:1034] running on GCE node
I20260806 11:40:42.961216 9267 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:40:42.961932 9267 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:40:42.963178 9267 hybrid_clock.cc:648] HybridClock initialized: now 1786016442963161 us; error 30 us; skew 500 ppm
I20260806 11:40:42.965540 9267 webserver.cc:460] Webserver started at http://127.21.14.194:39511/ using document root <none> and password file <none>
I20260806 11:40:42.966197 9267 fs_manager.cc:359] Metadata directory not provided
I20260806 11:40:42.966300 9267 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:40:42.966570 9267 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:40:42.968923 9267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786016169045816-21563-0/minicluster-data/ts-1/data/instance:
uuid: "5b80f6bd01404ac0930c3eb47e15e110"
format_stamp: "Formatted at 2026-08-06 11:40:42 on dist-test-slave-7c35"
server_key: "1624598e667d2593af29a386d64df018"
server_key_iv: "de8523db07076200be551d8214ff3c2a"
server_key_version: "encryptionkey@0"
I20260806 11:40:42.969565 9267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786016169045816-21563-0/minicluster-data/ts-1/wal/instance:
uuid: "5b80f6bd01404ac0930c3eb47e15e110"
format_stamp: "Formatted at 2026-08-06 11:40:42 on dist-test-slave-7c35"
server_key: "1624598e667d2593af29a386d64df018"
server_key_iv: "de8523db07076200be551d8214ff3c2a"
server_key_version: "encryptionkey@0"
I20260806 11:40:42.973873 9267 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 11:40:42.976637 9281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:42.978147 9267 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 11:40:42.978333 9267 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786016169045816-21563-0/minicluster-data/ts-1/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786016169045816-21563-0/minicluster-data/ts-1/wal
uuid: "5b80f6bd01404ac0930c3eb47e15e110"
format_stamp: "Formatted at 2026-08-06 11:40:42 on dist-test-slave-7c35"
server_key: "1624598e667d2593af29a386d64df018"
server_key_iv: "de8523db07076200be551d8214ff3c2a"
server_key_version: "encryptionkey@0"
I20260806 11:40:42.978483 9267 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786016169045816-21563-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786016169045816-21563-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786016169045816-21563-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:40:43.016487 9267 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:40:43.017306 9267 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:40:43.017553 9267 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:40:43.018239 9267 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:40:43.019441 9267 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:40:43.019526 9267 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:43.019610 9267 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:40:43.019668 9267 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:43.029621 9267 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.194:37947
I20260806 11:40:43.029651 9402 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.194:37947 every 8 connection(s)
I20260806 11:40:43.030717 9267 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
I20260806 11:40:43.032384 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 9267
I20260806 11:40:43.032526 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786016169045816-21563-0/minicluster-data/ts-1/wal/instance
I20260806 11:40:43.032914 21563 external_mini_cluster.cc:1428] Setting key 3c0e73a44c570fb9850389acfc67da32
I20260806 11:40:43.035133 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786016169045816-21563-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786016169045816-21563-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786016169045816-21563-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.195:0
--local_ip_for_outbound_sockets=127.21.14.195
--webserver_interface=127.21.14.195
--webserver_port=0
--tserver_master_addrs=127.21.14.254:34097
--builtin_ntp_servers=127.21.14.212:40717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 11:40:43.040963 9101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:43.032870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:42105 (local address 127.21.14.254:34097)
0806 11:40:43.033033 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:43.033037 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:43.034314 (+ 1277us) server_negotiation.cc:407] Connection header received
0806 11:40:43.035977 (+ 1663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:43.035981 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:43.036038 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:43.036144 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:40:43.037096 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:43.037654 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:43.038878 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:43.039112 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:43.039226 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:43.040442 (+ 1216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:40:43.040445 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:40:43.040447 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 11:40:43.040461 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:40:43.040479 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 11:40:43.040487 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:40:43.040562 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:43.040744 (+ 182us) server_negotiation.cc:299] Negotiation successful
0806 11:40:43.040811 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 11:40:43.041851 9405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:43.033425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:34097 (local address 127.21.14.194:42105)
0806 11:40:43.034173 (+ 748us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:43.034228 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 11:40:43.035747 (+ 1519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:40:43.036331 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:40:43.036339 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:40:43.036427 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:40:43.036948 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:43.036961 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:43.037797 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:43.037800 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:43.038715 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:43.038725 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:43.039278 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:43.039283 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:43.039456 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:43.040158 (+ 702us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 11:40:43.040160 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:40:43.040202 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 11:40:43.040273 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:40:43.040578 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:40:43.040595 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:40:43.040600 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 11:40:43.040728 (+ 128us) client_negotiation.cc:240] Negotiation successful
0806 11:40:43.041044 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":147,"threads_started":1}
I20260806 11:40:43.043200 9403 heartbeater.cc:347] Connected to a master server at 127.21.14.254:34097
I20260806 11:40:43.043510 9403 heartbeater.cc:464] Registering TS with master...
I20260806 11:40:43.044273 9403 heartbeater.cc:511] Master 127.21.14.254:34097 requested a full tablet report, sending...
I20260806 11:40:43.045605 9051 ts_manager.cc:194] Registered new tserver with Master: 5b80f6bd01404ac0930c3eb47e15e110 (127.21.14.194:37947)
I20260806 11:40:43.046209 9051 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.14.194:42105
W20260806 11:40:43.160903 9406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:40:43.161238 9406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:40:43.161285 9406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:40:43.165884 9406 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:40:43.165967 9406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:40:43.166065 9406 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.195
I20260806 11:40:43.171379 9406 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:40717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786016169045816-21563-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786016169045816-21563-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.14.195
--webserver_port=0
--tserver_master_addrs=127.21.14.254:34097
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9406
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.195
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786016169045816-21563-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:40:43.172725 9406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:40:43.174259 9406 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:40:43.185204 9412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:40:43.185510 9406 server_base.cc:1034] running on GCE node
W20260806 11:40:43.185199 9414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:40:43.185204 9411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:40:43.186241 9406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:40:43.187088 9406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:40:43.188299 9406 hybrid_clock.cc:648] HybridClock initialized: now 1786016443188283 us; error 29 us; skew 500 ppm
I20260806 11:40:43.190541 9406 webserver.cc:460] Webserver started at http://127.21.14.195:45331/ using document root <none> and password file <none>
I20260806 11:40:43.191282 9406 fs_manager.cc:359] Metadata directory not provided
I20260806 11:40:43.191386 9406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:40:43.191634 9406 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:40:43.193877 9406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786016169045816-21563-0/minicluster-data/ts-2/data/instance:
uuid: "7c174eab700442fcabeeafdfaac62332"
format_stamp: "Formatted at 2026-08-06 11:40:43 on dist-test-slave-7c35"
server_key: "d24cbc05d4b92bd185ecdfa266f41674"
server_key_iv: "64d52763cd70e8656d7df2feeea018b8"
server_key_version: "encryptionkey@0"
I20260806 11:40:43.194495 9406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786016169045816-21563-0/minicluster-data/ts-2/wal/instance:
uuid: "7c174eab700442fcabeeafdfaac62332"
format_stamp: "Formatted at 2026-08-06 11:40:43 on dist-test-slave-7c35"
server_key: "d24cbc05d4b92bd185ecdfa266f41674"
server_key_iv: "64d52763cd70e8656d7df2feeea018b8"
server_key_version: "encryptionkey@0"
I20260806 11:40:43.198575 9406 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260806 11:40:43.201396 9420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:43.203004 9406 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 11:40:43.203155 9406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786016169045816-21563-0/minicluster-data/ts-2/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786016169045816-21563-0/minicluster-data/ts-2/wal
uuid: "7c174eab700442fcabeeafdfaac62332"
format_stamp: "Formatted at 2026-08-06 11:40:43 on dist-test-slave-7c35"
server_key: "d24cbc05d4b92bd185ecdfa266f41674"
server_key_iv: "64d52763cd70e8656d7df2feeea018b8"
server_key_version: "encryptionkey@0"
I20260806 11:40:43.203292 9406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786016169045816-21563-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786016169045816-21563-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786016169045816-21563-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:40:43.216069 9406 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:40:43.216985 9406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:40:43.217217 9406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:40:43.217903 9406 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:40:43.219022 9406 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:40:43.219098 9406 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:43.219175 9406 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:40:43.219228 9406 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:43.229095 9406 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.195:46203
I20260806 11:40:43.229125 9539 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.195:46203 every 8 connection(s)
I20260806 11:40:43.230077 9406 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
I20260806 11:40:43.233291 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 9406
I20260806 11:40:43.233434 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786016169045816-21563-0/minicluster-data/ts-2/wal/instance
I20260806 11:40:43.233738 21563 external_mini_cluster.cc:1428] Setting key f866962ffe9301fbafc6f5884cde3c5e
I20260806 11:40:43.240357 9101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:43.232136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:43289 (local address 127.21.14.254:34097)
0806 11:40:43.232282 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:43.232286 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:43.233533 (+ 1247us) server_negotiation.cc:407] Connection header received
0806 11:40:43.235325 (+ 1792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:43.235328 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:43.235416 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:43.235520 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:40:43.236675 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:43.237234 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:43.238407 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:43.238631 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:43.238717 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:43.239916 (+ 1199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:40:43.239919 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:40:43.239921 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 11:40:43.239936 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:40:43.239956 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 11:40:43.239964 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:40:43.240027 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:43.240178 (+ 151us) server_negotiation.cc:299] Negotiation successful
0806 11:40:43.240236 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 11:40:43.241289 9542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:43.232602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:34097 (local address 127.21.14.195:43289)
0806 11:40:43.233333 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:43.233409 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0806 11:40:43.235040 (+ 1631us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:40:43.235735 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:40:43.235744 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:40:43.235859 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:40:43.236526 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:43.236543 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:43.237357 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:43.237360 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:43.238286 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:43.238295 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:43.238773 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:43.238776 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:43.238996 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:43.239648 (+ 652us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 11:40:43.239650 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:40:43.239692 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 11:40:43.239761 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:40:43.240050 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:40:43.240057 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:40:43.240061 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 11:40:43.240160 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 11:40:43.240414 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":125,"threads_started":1}
I20260806 11:40:43.242602 9540 heartbeater.cc:347] Connected to a master server at 127.21.14.254:34097
I20260806 11:40:43.242887 9540 heartbeater.cc:464] Registering TS with master...
I20260806 11:40:43.243469 9540 heartbeater.cc:511] Master 127.21.14.254:34097 requested a full tablet report, sending...
I20260806 11:40:43.244620 9051 ts_manager.cc:194] Registered new tserver with Master: 7c174eab700442fcabeeafdfaac62332 (127.21.14.195:46203)
I20260806 11:40:43.245203 9051 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.14.195:43289
I20260806 11:40:43.248791 21563 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 11:40:43.255967 9101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:43.251177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48676 (local address 127.21.14.254:34097)
0806 11:40:43.251496 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:43.251501 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:43.251679 (+ 178us) server_negotiation.cc:407] Connection header received
0806 11:40:43.252008 (+ 329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:43.252011 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:43.252062 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:43.252176 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:40:43.252538 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:43.253089 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:43.254019 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:43.254251 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:43.254365 (+ 114us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:43.255411 (+ 1046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:40:43.255415 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:40:43.255417 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 11:40:43.255431 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:40:43.255451 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 11:40:43.255459 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:40:43.255599 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:43.255753 (+ 154us) server_negotiation.cc:299] Negotiation successful
0806 11:40:43.255802 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139}
I20260806 11:40:43.259377 9051 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48676:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 11:40:43.262650 9051 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 11:40:43.278352 9551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:43.272069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.195:46203 (local address 127.0.0.1:45504)
0806 11:40:43.272972 (+ 903us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:43.272987 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 11:40:43.273074 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:40:43.273909 (+ 835us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:40:43.273912 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:40:43.273926 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:40:43.274018 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:43.274024 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:43.275216 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:43.275221 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:43.276756 (+ 1535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:43.276764 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:43.277768 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:43.277772 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:43.277944 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:43.277988 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 11:40:43.278094 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 11:40:43.278204 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":821,"mutex_wait_us":34,"thread_start_us":97,"threads_started":1}
I20260806 11:40:43.278474 9555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:43.272794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56686 (local address 127.21.14.193:43465)
0806 11:40:43.273649 (+ 855us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:43.273655 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:43.274196 (+ 541us) server_negotiation.cc:407] Connection header received
0806 11:40:43.274355 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:43.274360 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:43.274534 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:43.274681 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:40:43.274933 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:43.275833 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:43.277180 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:43.277842 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:43.277951 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:43.278157 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:43.278209 (+ 52us) server_negotiation.cc:299] Negotiation successful
0806 11:40:43.278315 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":753,"thread_start_us":135,"threads_started":1}
I20260806 11:40:43.278724 9552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:43.271876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.194:37947 (local address 127.0.0.1:54680)
0806 11:40:43.272462 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:43.272516 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 11:40:43.272646 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:40:43.273801 (+ 1155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:40:43.273808 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:40:43.273832 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:40:43.274017 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:43.274024 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:43.275458 (+ 1434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:43.275461 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:43.276607 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:43.276618 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:43.277751 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:43.277755 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:43.278409 (+ 654us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:43.278431 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 11:40:43.278505 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 11:40:43.278590 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":404,"mutex_wait_us":492,"thread_start_us":68,"threads_started":1}
I20260806 11:40:43.278366 9554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:43.272593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.193:43465 (local address 127.0.0.1:56686)
0806 11:40:43.274092 (+ 1499us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:43.274105 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 11:40:43.274206 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:40:43.274684 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:40:43.274688 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:40:43.274706 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:40:43.274789 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:43.274794 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:43.275946 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:43.275951 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:43.277051 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:43.277060 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:43.277969 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:43.277972 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:43.278046 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:43.278069 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 11:40:43.278133 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 11:40:43.278213 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1423,"thread_start_us":104,"threads_started":1}
I20260806 11:40:43.279327 9553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:43.272549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54680 (local address 127.21.14.194:37947)
0806 11:40:43.273153 (+ 604us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:43.273167 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:43.273197 (+ 30us) server_negotiation.cc:407] Connection header received
0806 11:40:43.273284 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:43.273290 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:43.273493 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:43.273683 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:40:43.274164 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:43.275338 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:43.276747 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:43.277590 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:43.277745 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:43.277980 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:43.279054 (+ 1074us) server_negotiation.cc:299] Negotiation successful
0806 11:40:43.279168 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":489,"thread_start_us":123,"threads_started":1}
I20260806 11:40:43.280400 9556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:43.273044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45504 (local address 127.21.14.195:46203)
0806 11:40:43.273461 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:43.273472 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:43.273497 (+ 25us) server_negotiation.cc:407] Connection header received
0806 11:40:43.273575 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:43.273580 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:43.273772 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:43.273940 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:40:43.274106 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:43.275107 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:43.276879 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:43.277611 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:43.279142 (+ 1531us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:43.279342 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:43.279401 (+ 59us) server_negotiation.cc:299] Negotiation successful
0806 11:40:43.280226 (+ 825us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":179,"threads_started":1}
I20260806 11:40:43.282084 9194 tablet_service.cc:1467] Processing CreateTablet for tablet 0eb72d004cb94214afb989c8bed6160d (DEFAULT_TABLE table=test-table [id=ed753ba8bcd8416ca1a18d7c168aa9c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:40:43.282269 9335 tablet_service.cc:1467] Processing CreateTablet for tablet 0eb72d004cb94214afb989c8bed6160d (DEFAULT_TABLE table=test-table [id=ed753ba8bcd8416ca1a18d7c168aa9c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:40:43.283268 9194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0eb72d004cb94214afb989c8bed6160d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:40:43.283399 9335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0eb72d004cb94214afb989c8bed6160d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:40:43.283110 9473 tablet_service.cc:1467] Processing CreateTablet for tablet 0eb72d004cb94214afb989c8bed6160d (DEFAULT_TABLE table=test-table [id=ed753ba8bcd8416ca1a18d7c168aa9c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:40:43.284106 9473 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0eb72d004cb94214afb989c8bed6160d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:40:43.290140 9557 tablet_bootstrap.cc:492] T 0eb72d004cb94214afb989c8bed6160d P 498c59dba5c44ceb92843987b108f5cb: Bootstrap starting.
I20260806 11:40:43.291760 9559 tablet_bootstrap.cc:492] T 0eb72d004cb94214afb989c8bed6160d P 7c174eab700442fcabeeafdfaac62332: Bootstrap starting.
I20260806 11:40:43.293423 9557 tablet_bootstrap.cc:654] T 0eb72d004cb94214afb989c8bed6160d P 498c59dba5c44ceb92843987b108f5cb: Neither blocks nor log segments found. Creating new log.
I20260806 11:40:43.294134 9559 tablet_bootstrap.cc:654] T 0eb72d004cb94214afb989c8bed6160d P 7c174eab700442fcabeeafdfaac62332: Neither blocks nor log segments found. Creating new log.
I20260806 11:40:43.294406 9557 log.cc:824] T 0eb72d004cb94214afb989c8bed6160d P 498c59dba5c44ceb92843987b108f5cb: Log is configured to *not* fsync() on all Append() calls
I20260806 11:40:43.295039 9559 log.cc:824] T 0eb72d004cb94214afb989c8bed6160d P 7c174eab700442fcabeeafdfaac62332: Log is configured to *not* fsync() on all Append() calls
I20260806 11:40:43.295328 9558 tablet_bootstrap.cc:492] T 0eb72d004cb94214afb989c8bed6160d P 5b80f6bd01404ac0930c3eb47e15e110: Bootstrap starting.
I20260806 11:40:43.296767 9557 tablet_bootstrap.cc:492] T 0eb72d004cb94214afb989c8bed6160d P 498c59dba5c44ceb92843987b108f5cb: No bootstrap required, opened a new log
I20260806 11:40:43.296993 9557 ts_tablet_manager.cc:1397] T 0eb72d004cb94214afb989c8bed6160d P 498c59dba5c44ceb92843987b108f5cb: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.003s
I20260806 11:40:43.297065 9559 tablet_bootstrap.cc:492] T 0eb72d004cb94214afb989c8bed6160d P 7c174eab700442fcabeeafdfaac62332: No bootstrap required, opened a new log
I20260806 11:40:43.297250 9559 ts_tablet_manager.cc:1397] T 0eb72d004cb94214afb989c8bed6160d P 7c174eab700442fcabeeafdfaac62332: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260806 11:40:43.297775 9558 tablet_bootstrap.cc:654] T 0eb72d004cb94214afb989c8bed6160d P 5b80f6bd01404ac0930c3eb47e15e110: Neither blocks nor log segments found. Creating new log.
I20260806 11:40:43.298637 9558 log.cc:824] T 0eb72d004cb94214afb989c8bed6160d P 5b80f6bd01404ac0930c3eb47e15e110: Log is configured to *not* fsync() on all Append() calls
I20260806 11:40:43.301102 9558 tablet_bootstrap.cc:492] T 0eb72d004cb94214afb989c8bed6160d P 5b80f6bd01404ac0930c3eb47e15e110: No bootstrap required, opened a new log
I20260806 11:40:43.300956 9557 raft_consensus.cc:348] T 0eb72d004cb94214afb989c8bed6160d P 498c59dba5c44ceb92843987b108f5cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b80f6bd01404ac0930c3eb47e15e110" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37947 } } peers { permanent_uuid: "7c174eab700442fcabeeafdfaac62332" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 46203 } } peers { permanent_uuid: "498c59dba5c44ceb92843987b108f5cb" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 43465 } }
I20260806 11:40:43.301225 9557 raft_consensus.cc:374] T 0eb72d004cb94214afb989c8bed6160d P 498c59dba5c44ceb92843987b108f5cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:40:43.301288 9557 raft_consensus.cc:729] T 0eb72d004cb94214afb989c8bed6160d P 498c59dba5c44ceb92843987b108f5cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 498c59dba5c44ceb92843987b108f5cb, State: Initialized, Role: FOLLOWER
I20260806 11:40:43.301319 9558 ts_tablet_manager.cc:1397] T 0eb72d004cb94214afb989c8bed6160d P 5b80f6bd01404ac0930c3eb47e15e110: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 11:40:43.301525 9559 raft_consensus.cc:348] T 0eb72d004cb94214afb989c8bed6160d P 7c174eab700442fcabeeafdfaac62332 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b80f6bd01404ac0930c3eb47e15e110" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37947 } } peers { permanent_uuid: "7c174eab700442fcabeeafdfaac62332" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 46203 } } peers { permanent_uuid: "498c59dba5c44ceb92843987b108f5cb" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 43465 } }
I20260806 11:40:43.301826 9559 raft_consensus.cc:374] T 0eb72d004cb94214afb989c8bed6160d P 7c174eab700442fcabeeafdfaac62332 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:40:43.301772 9557 consensus_queue.cc:260] T 0eb72d004cb94214afb989c8bed6160d P 498c59dba5c44ceb92843987b108f5cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b80f6bd01404ac0930c3eb47e15e110" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37947 } } peers { permanent_uuid: "7c174eab700442fcabeeafdfaac62332" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 46203 } } peers { permanent_uuid: "498c59dba5c44ceb92843987b108f5cb" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 43465 } }
I20260806 11:40:43.301916 9559 raft_consensus.cc:729] T 0eb72d004cb94214afb989c8bed6160d P 7c174eab700442fcabeeafdfaac62332 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c174eab700442fcabeeafdfaac62332, State: Initialized, Role: FOLLOWER
I20260806 11:40:43.302359 9559 consensus_queue.cc:260] T 0eb72d004cb94214afb989c8bed6160d P 7c174eab700442fcabeeafdfaac62332 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b80f6bd01404ac0930c3eb47e15e110" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37947 } } peers { permanent_uuid: "7c174eab700442fcabeeafdfaac62332" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 46203 } } peers { permanent_uuid: "498c59dba5c44ceb92843987b108f5cb" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 43465 } }
I20260806 11:40:43.303413 9557 ts_tablet_manager.cc:1428] T 0eb72d004cb94214afb989c8bed6160d P 498c59dba5c44ceb92843987b108f5cb: Time spent starting tablet: real 0.006s user 0.002s sys 0.005s
I20260806 11:40:43.303411 9559 ts_tablet_manager.cc:1428] T 0eb72d004cb94214afb989c8bed6160d P 7c174eab700442fcabeeafdfaac62332: Time spent starting tablet: real 0.006s user 0.006s sys 0.002s
I20260806 11:40:43.303815 9263 heartbeater.cc:503] Master 127.21.14.254:34097 was elected leader, sending a full tablet report...
I20260806 11:40:43.304790 9540 heartbeater.cc:503] Master 127.21.14.254:34097 was elected leader, sending a full tablet report...
I20260806 11:40:43.305565 9558 raft_consensus.cc:348] T 0eb72d004cb94214afb989c8bed6160d P 5b80f6bd01404ac0930c3eb47e15e110 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b80f6bd01404ac0930c3eb47e15e110" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37947 } } peers { permanent_uuid: "7c174eab700442fcabeeafdfaac62332" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 46203 } } peers { permanent_uuid: "498c59dba5c44ceb92843987b108f5cb" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 43465 } }
I20260806 11:40:43.305858 9558 raft_consensus.cc:374] T 0eb72d004cb94214afb989c8bed6160d P 5b80f6bd01404ac0930c3eb47e15e110 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:40:43.305929 9558 raft_consensus.cc:729] T 0eb72d004cb94214afb989c8bed6160d P 5b80f6bd01404ac0930c3eb47e15e110 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b80f6bd01404ac0930c3eb47e15e110, State: Initialized, Role: FOLLOWER
W20260806 11:40:43.305994 9264 tablet.cc:2369] T 0eb72d004cb94214afb989c8bed6160d P 498c59dba5c44ceb92843987b108f5cb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 11:40:43.306434 9558 consensus_queue.cc:260] T 0eb72d004cb94214afb989c8bed6160d P 5b80f6bd01404ac0930c3eb47e15e110 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b80f6bd01404ac0930c3eb47e15e110" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37947 } } peers { permanent_uuid: "7c174eab700442fcabeeafdfaac62332" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 46203 } } peers { permanent_uuid: "498c59dba5c44ceb92843987b108f5cb" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 43465 } }
I20260806 11:40:43.307293 9403 heartbeater.cc:503] Master 127.21.14.254:34097 was elected leader, sending a full tablet report...
I20260806 11:40:43.308837 9558 ts_tablet_manager.cc:1428] T 0eb72d004cb94214afb989c8bed6160d P 5b80f6bd01404ac0930c3eb47e15e110: Time spent starting tablet: real 0.007s user 0.007s sys 0.001s
I20260806 11:40:43.471292 9564 raft_consensus.cc:482] T 0eb72d004cb94214afb989c8bed6160d P 7c174eab700442fcabeeafdfaac62332 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 11:40:43.471534 9564 raft_consensus.cc:504] T 0eb72d004cb94214afb989c8bed6160d P 7c174eab700442fcabeeafdfaac62332 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b80f6bd01404ac0930c3eb47e15e110" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37947 } } peers { permanent_uuid: "7c174eab700442fcabeeafdfaac62332" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 46203 } } peers { permanent_uuid: "498c59dba5c44ceb92843987b108f5cb" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 43465 } }
I20260806 11:40:43.472718 9564 leader_election.cc:288] T 0eb72d004cb94214afb989c8bed6160d P 7c174eab700442fcabeeafdfaac62332 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b80f6bd01404ac0930c3eb47e15e110 (127.21.14.194:37947), 498c59dba5c44ceb92843987b108f5cb (127.21.14.193:43465)
I20260806 11:40:43.476879 9553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:43.473191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:40707 (local address 127.21.14.194:37947)
0806 11:40:43.473344 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:43.473351 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:43.473369 (+ 18us) server_negotiation.cc:407] Connection header received
0806 11:40:43.473587 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:43.473593 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:43.473659 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:43.473748 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:40:43.474072 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:43.474604 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:43.475766 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:43.476278 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:43.476432 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:43.476484 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:43.476672 (+ 188us) server_negotiation.cc:299] Negotiation successful
0806 11:40:43.476747 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 11:40:43.476886 9542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:43.473106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.194:37947 (local address 127.21.14.195:40707)
0806 11:40:43.473280 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:43.473299 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 11:40:43.473417 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:40:43.473792 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:40:43.473795 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:40:43.473815 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:40:43.473945 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:43.473952 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:43.474724 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:43.474729 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:43.475630 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:43.475639 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:43.476424 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:43.476428 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:43.476533 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:43.476555 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 11:40:43.476656 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 11:40:43.476734 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53,"mutex_wait_us":8}
I20260806 11:40:43.477126 9566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:43.473437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.193:43465 (local address 127.21.14.195:43519)
0806 11:40:43.473820 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:43.473836 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 11:40:43.473940 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:40:43.474195 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:40:43.474198 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:40:43.474214 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:40:43.474351 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:43.474357 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:43.475194 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:43.475198 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:43.475970 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:43.475978 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:43.476677 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:43.476681 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:43.476783 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:43.476805 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 11:40:43.476904 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 11:40:43.476974 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":149,"threads_started":1}
I20260806 11:40:43.477113 9555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:43.473643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:43519 (local address 127.21.14.193:43465)
0806 11:40:43.473811 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:43.473816 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:43.473899 (+ 83us) server_negotiation.cc:407] Connection header received
0806 11:40:43.474032 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:43.474035 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:43.474088 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:43.474173 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:40:43.474471 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:43.475059 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:43.476131 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:43.476539 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:43.476682 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:43.476731 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:43.476923 (+ 192us) server_negotiation.cc:299] Negotiation successful
0806 11:40:43.476997 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260806 11:40:43.477761 9355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eb72d004cb94214afb989c8bed6160d" candidate_uuid: "7c174eab700442fcabeeafdfaac62332" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b80f6bd01404ac0930c3eb47e15e110" is_pre_election: true
I20260806 11:40:43.478030 9216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eb72d004cb94214afb989c8bed6160d" candidate_uuid: "7c174eab700442fcabeeafdfaac62332" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "498c59dba5c44ceb92843987b108f5cb" is_pre_election: true
I20260806 11:40:43.478138 9355 raft_consensus.cc:2454] T 0eb72d004cb94214afb989c8bed6160d P 5b80f6bd01404ac0930c3eb47e15e110 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c174eab700442fcabeeafdfaac62332 in term 0.
I20260806 11:40:43.478355 9216 raft_consensus.cc:2454] T 0eb72d004cb94214afb989c8bed6160d P 498c59dba5c44ceb92843987b108f5cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c174eab700442fcabeeafdfaac62332 in term 0.
I20260806 11:40:43.478741 9421 leader_election.cc:302] T 0eb72d004cb94214afb989c8bed6160d P 7c174eab700442fcabeeafdfaac62332 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b80f6bd01404ac0930c3eb47e15e110, 7c174eab700442fcabeeafdfaac62332; no voters:
I20260806 11:40:43.479177 9564 raft_consensus.cc:2781] T 0eb72d004cb94214afb989c8bed6160d P 7c174eab700442fcabeeafdfaac62332 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 11:40:43.479308 9564 raft_consensus.cc:482] T 0eb72d004cb94214afb989c8bed6160d P 7c174eab700442fcabeeafdfaac62332 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 11:40:43.479436 9564 raft_consensus.cc:3037] T 0eb72d004cb94214afb989c8bed6160d P 7c174eab700442fcabeeafdfaac62332 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:40:43.480635 9564 raft_consensus.cc:504] T 0eb72d004cb94214afb989c8bed6160d P 7c174eab700442fcabeeafdfaac62332 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b80f6bd01404ac0930c3eb47e15e110" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37947 } } peers { permanent_uuid: "7c174eab700442fcabeeafdfaac62332" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 46203 } } peers { permanent_uuid: "498c59dba5c44ceb92843987b108f5cb" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 43465 } }
I20260806 11:40:43.481086 9564 leader_election.cc:288] T 0eb72d004cb94214afb989c8bed6160d P 7c174eab700442fcabeeafdfaac62332 [CANDIDATE]: Term 1 election: Requested vote from peers 5b80f6bd01404ac0930c3eb47e15e110 (127.21.14.194:37947), 498c59dba5c44ceb92843987b108f5cb (127.21.14.193:43465)
W20260806 11:40:43.481129 9541 tablet.cc:2369] T 0eb72d004cb94214afb989c8bed6160d P 7c174eab700442fcabeeafdfaac62332: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 11:40:43.481520 9355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eb72d004cb94214afb989c8bed6160d" candidate_uuid: "7c174eab700442fcabeeafdfaac62332" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b80f6bd01404ac0930c3eb47e15e110"
I20260806 11:40:43.481557 9216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eb72d004cb94214afb989c8bed6160d" candidate_uuid: "7c174eab700442fcabeeafdfaac62332" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "498c59dba5c44ceb92843987b108f5cb"
I20260806 11:40:43.481704 9355 raft_consensus.cc:3037] T 0eb72d004cb94214afb989c8bed6160d P 5b80f6bd01404ac0930c3eb47e15e110 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:40:43.481704 9216 raft_consensus.cc:3037] T 0eb72d004cb94214afb989c8bed6160d P 498c59dba5c44ceb92843987b108f5cb [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:40:43.483196 9216 raft_consensus.cc:2454] T 0eb72d004cb94214afb989c8bed6160d P 498c59dba5c44ceb92843987b108f5cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c174eab700442fcabeeafdfaac62332 in term 1.
I20260806 11:40:43.483196 9355 raft_consensus.cc:2454] T 0eb72d004cb94214afb989c8bed6160d P 5b80f6bd01404ac0930c3eb47e15e110 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c174eab700442fcabeeafdfaac62332 in term 1.
I20260806 11:40:43.483582 9421 leader_election.cc:302] T 0eb72d004cb94214afb989c8bed6160d P 7c174eab700442fcabeeafdfaac62332 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 498c59dba5c44ceb92843987b108f5cb, 7c174eab700442fcabeeafdfaac62332; no voters:
I20260806 11:40:43.483803 9564 raft_consensus.cc:2781] T 0eb72d004cb94214afb989c8bed6160d P 7c174eab700442fcabeeafdfaac62332 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:40:43.484077 9564 raft_consensus.cc:686] T 0eb72d004cb94214afb989c8bed6160d P 7c174eab700442fcabeeafdfaac62332 [term 1 LEADER]: Becoming Leader. State: Replica: 7c174eab700442fcabeeafdfaac62332, State: Running, Role: LEADER
I20260806 11:40:43.484453 9564 consensus_queue.cc:237] T 0eb72d004cb94214afb989c8bed6160d P 7c174eab700442fcabeeafdfaac62332 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b80f6bd01404ac0930c3eb47e15e110" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37947 } } peers { permanent_uuid: "7c174eab700442fcabeeafdfaac62332" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 46203 } } peers { permanent_uuid: "498c59dba5c44ceb92843987b108f5cb" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 43465 } }
I20260806 11:40:43.488047 9051 catalog_manager.cc:5495] T 0eb72d004cb94214afb989c8bed6160d P 7c174eab700442fcabeeafdfaac62332 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c174eab700442fcabeeafdfaac62332 (127.21.14.195). New cstate: current_term: 1 leader_uuid: "7c174eab700442fcabeeafdfaac62332" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b80f6bd01404ac0930c3eb47e15e110" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c174eab700442fcabeeafdfaac62332" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 46203 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "498c59dba5c44ceb92843987b108f5cb" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 43465 } health_report { overall_health: UNKNOWN } } }
I20260806 11:40:43.520488 9556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:43.515526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45514 (local address 127.21.14.195:46203)
0806 11:40:43.515681 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:43.515685 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:43.515706 (+ 21us) server_negotiation.cc:407] Connection header received
0806 11:40:43.515787 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:43.515789 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:43.515849 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:43.515950 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 11:40:43.516233 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:43.516783 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:43.517765 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:43.518148 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:43.518281 (+ 133us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:43.518895 (+ 614us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 11:40:43.519546 (+ 651us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 11:40:43.519708 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:43.520248 (+ 540us) server_negotiation.cc:299] Negotiation successful
0806 11:40:43.520303 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 11:40:43.527872 9216 raft_consensus.cc:1261] T 0eb72d004cb94214afb989c8bed6160d P 498c59dba5c44ceb92843987b108f5cb [term 1 FOLLOWER]: Refusing update from remote peer 7c174eab700442fcabeeafdfaac62332: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 11:40:43.527943 9355 raft_consensus.cc:1261] T 0eb72d004cb94214afb989c8bed6160d P 5b80f6bd01404ac0930c3eb47e15e110 [term 1 FOLLOWER]: Refusing update from remote peer 7c174eab700442fcabeeafdfaac62332: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 11:40:43.528692 9567 consensus_queue.cc:1035] T 0eb72d004cb94214afb989c8bed6160d P 7c174eab700442fcabeeafdfaac62332 [LEADER]: Connected to new peer: Peer: permanent_uuid: "498c59dba5c44ceb92843987b108f5cb" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 43465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 11:40:43.528973 9564 consensus_queue.cc:1035] T 0eb72d004cb94214afb989c8bed6160d P 7c174eab700442fcabeeafdfaac62332 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b80f6bd01404ac0930c3eb47e15e110" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 37947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 11:40:43.532001 9404 tablet.cc:2369] T 0eb72d004cb94214afb989c8bed6160d P 5b80f6bd01404ac0930c3eb47e15e110: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 11:40:43.540082 9572 mvcc.cc:205] Tried to move back new op lower bound from 7315523352681771008 to 7315523352515141632. Current Snapshot: MvccSnapshot[applied={T|T < 7315523352681771008}]
I20260806 11:40:43.549396 9051 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48676:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 11:40:43.549633 9051 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48676:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 11:40:43.552513 9051 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 396555bb61944261a44c0e82a7827248: Sending DeleteTablet for 3 replicas of tablet 0eb72d004cb94214afb989c8bed6160d
I20260806 11:40:43.553454 9473 tablet_service.cc:1514] Processing DeleteTablet for tablet 0eb72d004cb94214afb989c8bed6160d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:40:43 UTC) from {username='slave'} at 127.0.0.1:45504
I20260806 11:40:43.553684 9194 tablet_service.cc:1514] Processing DeleteTablet for tablet 0eb72d004cb94214afb989c8bed6160d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:40:43 UTC) from {username='slave'} at 127.0.0.1:56686
I20260806 11:40:43.553682 9335 tablet_service.cc:1514] Processing DeleteTablet for tablet 0eb72d004cb94214afb989c8bed6160d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:40:43 UTC) from {username='slave'} at 127.0.0.1:54680
I20260806 11:40:43.554055 9581 tablet_replica.cc:330] T 0eb72d004cb94214afb989c8bed6160d P 7c174eab700442fcabeeafdfaac62332: stopping tablet replica
I20260806 11:40:43.554445 9581 raft_consensus.cc:2229] T 0eb72d004cb94214afb989c8bed6160d P 7c174eab700442fcabeeafdfaac62332 [term 1 LEADER]: Raft consensus shutting down.
I20260806 11:40:43.554850 9582 tablet_replica.cc:330] T 0eb72d004cb94214afb989c8bed6160d P 5b80f6bd01404ac0930c3eb47e15e110: stopping tablet replica
I20260806 11:40:43.554955 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 9122
I20260806 11:40:43.554951 9581 raft_consensus.cc:2258] T 0eb72d004cb94214afb989c8bed6160d P 7c174eab700442fcabeeafdfaac62332 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 11:40:43.555888 9582 raft_consensus.cc:2229] T 0eb72d004cb94214afb989c8bed6160d P 5b80f6bd01404ac0930c3eb47e15e110 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 11:40:43.556284 9582 raft_consensus.cc:2258] T 0eb72d004cb94214afb989c8bed6160d P 5b80f6bd01404ac0930c3eb47e15e110 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 11:40:43.557787 9581 ts_tablet_manager.cc:1905] T 0eb72d004cb94214afb989c8bed6160d P 7c174eab700442fcabeeafdfaac62332: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 11:40:43.558280 9582 ts_tablet_manager.cc:1905] T 0eb72d004cb94214afb989c8bed6160d P 5b80f6bd01404ac0930c3eb47e15e110: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 11:40:43.560766 9582 ts_tablet_manager.cc:1918] T 0eb72d004cb94214afb989c8bed6160d P 5b80f6bd01404ac0930c3eb47e15e110: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 11:40:43.560909 9582 log.cc:1196] T 0eb72d004cb94214afb989c8bed6160d P 5b80f6bd01404ac0930c3eb47e15e110: Deleting WAL directory at /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786016169045816-21563-0/minicluster-data/ts-1/wal/wals/0eb72d004cb94214afb989c8bed6160d
I20260806 11:40:43.560885 9581 ts_tablet_manager.cc:1918] T 0eb72d004cb94214afb989c8bed6160d P 7c174eab700442fcabeeafdfaac62332: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 11:40:43.560981 9581 log.cc:1196] T 0eb72d004cb94214afb989c8bed6160d P 7c174eab700442fcabeeafdfaac62332: Deleting WAL directory at /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786016169045816-21563-0/minicluster-data/ts-2/wal/wals/0eb72d004cb94214afb989c8bed6160d
I20260806 11:40:43.561237 9582 ts_tablet_manager.cc:1939] T 0eb72d004cb94214afb989c8bed6160d P 5b80f6bd01404ac0930c3eb47e15e110: Deleting consensus metadata
I20260806 11:40:43.561304 9581 ts_tablet_manager.cc:1939] T 0eb72d004cb94214afb989c8bed6160d P 7c174eab700442fcabeeafdfaac62332: Deleting consensus metadata
I20260806 11:40:43.562704 9043 catalog_manager.cc:4841] TS 7c174eab700442fcabeeafdfaac62332 (127.21.14.195:46203): tablet 0eb72d004cb94214afb989c8bed6160d (table test-table [id=ed753ba8bcd8416ca1a18d7c168aa9c4]) successfully deleted
I20260806 11:40:43.562722 9041 catalog_manager.cc:4841] TS 5b80f6bd01404ac0930c3eb47e15e110 (127.21.14.194:37947): tablet 0eb72d004cb94214afb989c8bed6160d (table test-table [id=ed753ba8bcd8416ca1a18d7c168aa9c4]) successfully deleted
W20260806 11:40:43.567695 9041 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.14.193:43465 (error 108)
I20260806 11:40:43.568006 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 9267
I20260806 11:40:43.569607 9552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:43.569056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.193:43465 (local address 127.0.0.1:56688)
0806 11:40:43.569365 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:43.569468 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.193:43465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170}
W20260806 11:40:43.569945 9041 catalog_manager.cc:4584] TS 498c59dba5c44ceb92843987b108f5cb (127.21.14.193:43465): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0eb72d004cb94214afb989c8bed6160d: Network error: Client connection negotiation failed: client connection to 127.21.14.193:43465: connect: Connection refused (error 111)
I20260806 11:40:43.577379 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 9406
I20260806 11:40:43.585505 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 9026
2026-08-06T11:40:43Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1293 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 11:40:43 dist-test-slave-7c35 krb5kdc[9587](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 11:40:43 dist-test-slave-7c35 krb5kdc[9587](info): set up 2 sockets
Aug 06 11:40:43 dist-test-slave-7c35 krb5kdc[9587](info): commencing operation
krb5kdc: starting...
W20260806 11:40:44.690623 21563 mini_kdc.cc:121] Time spent starting KDC: real 1.079s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Aug 06 11:40:44 dist-test-slave-7c35 krb5kdc[9587](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016444, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T11:40:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T11:40:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.14.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab.
Entry for principal kudu/127.21.14.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab.
I20260806 11:40:44.830503 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:35875
--webserver_interface=127.21.14.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab
--principal=kudu/127.21.14.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:35921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.14.254:35875
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:40:44.949244 9604 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:40:44.949604 9604 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:40:44.949687 9604 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:40:44.953871 9604 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 11:40:44.953985 9604 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:40:44.954022 9604 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:40:44.954061 9604 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 11:40:44.954097 9604 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 11:40:44.959656 9604 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:35921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.14.254:35875
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab
--principal=kudu/127.21.14.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:35875
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.14.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9604
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:40:44.960850 9604 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:40:44.961962 9604 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:40:44.971323 9609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:40:44.971319 9610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:40:44.971323 9613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:40:44.971714 9604 server_base.cc:1034] running on GCE node
I20260806 11:40:44.972388 9604 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:40:44.973608 9604 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:40:44.974808 9604 hybrid_clock.cc:648] HybridClock initialized: now 1786016444974795 us; error 35 us; skew 500 ppm
Aug 06 11:40:44 dist-test-slave-7c35 krb5kdc[9587](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016444, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:40:44.978358 9604 init.cc:377] Logged in from keytab as kudu/127.21.14.254@KRBTEST.COM (short username kudu)
I20260806 11:40:44.979529 9604 webserver.cc:460] Webserver started at http://127.21.14.254:41819/ using document root <none> and password file <none>
I20260806 11:40:44.980168 9604 fs_manager.cc:359] Metadata directory not provided
I20260806 11:40:44.980235 9604 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:40:44.980486 9604 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:40:44.982687 9604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/minicluster-data/master-0/data/instance:
uuid: "ed12a18ee96940db986f2da10542f0fe"
format_stamp: "Formatted at 2026-08-06 11:40:44 on dist-test-slave-7c35"
server_key: "d1a3076602c835bfe83e8559d475c2d6"
server_key_iv: "770a1e84d4f9f21e01668cc6207a39cf"
server_key_version: "encryptionkey@0"
I20260806 11:40:44.983351 9604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/minicluster-data/master-0/wal/instance:
uuid: "ed12a18ee96940db986f2da10542f0fe"
format_stamp: "Formatted at 2026-08-06 11:40:44 on dist-test-slave-7c35"
server_key: "d1a3076602c835bfe83e8559d475c2d6"
server_key_iv: "770a1e84d4f9f21e01668cc6207a39cf"
server_key_version: "encryptionkey@0"
I20260806 11:40:44.987424 9604 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 11:40:44.990142 9620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:44.991611 9604 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 11:40:44.991777 9604 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/minicluster-data/master-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/minicluster-data/master-0/wal
uuid: "ed12a18ee96940db986f2da10542f0fe"
format_stamp: "Formatted at 2026-08-06 11:40:44 on dist-test-slave-7c35"
server_key: "d1a3076602c835bfe83e8559d475c2d6"
server_key_iv: "770a1e84d4f9f21e01668cc6207a39cf"
server_key_version: "encryptionkey@0"
I20260806 11:40:44.991911 9604 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:40:45.023686 9604 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:40:45.027709 9604 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:40:45.027982 9604 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:40:45.036705 9604 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.254:35875
I20260806 11:40:45.036720 9672 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.254:35875 every 8 connection(s)
I20260806 11:40:45.037828 9604 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
I20260806 11:40:45.038825 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 9604
I20260806 11:40:45.039021 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/minicluster-data/master-0/wal/instance
I20260806 11:40:45.039337 21563 external_mini_cluster.cc:1428] Setting key fb892d4c28e21f95c214af73fe5fe8fc
I20260806 11:40:45.041354 9673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:40:45.048763 9673 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed12a18ee96940db986f2da10542f0fe: Bootstrap starting.
Aug 06 11:40:45 dist-test-slave-7c35 krb5kdc[9587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016444, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:40:45.051753 9673 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ed12a18ee96940db986f2da10542f0fe: Neither blocks nor log segments found. Creating new log.
I20260806 11:40:45.052805 9673 log.cc:824] T 00000000000000000000000000000000 P ed12a18ee96940db986f2da10542f0fe: Log is configured to *not* fsync() on all Append() calls
I20260806 11:40:45.055522 9673 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed12a18ee96940db986f2da10542f0fe: No bootstrap required, opened a new log
I20260806 11:40:45.056454 9676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:45.040934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49458 (local address 127.21.14.254:35875)
0806 11:40:45.041417 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:45.041438 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:45.041485 (+ 47us) server_negotiation.cc:407] Connection header received
0806 11:40:45.042357 (+ 872us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:45.042384 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:45.042830 (+ 446us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:45.043276 (+ 446us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:40:45.044162 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:45.045735 (+ 1573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:45.046711 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:45.047499 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:45.047674 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:45.050719 (+ 3045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:40:45.050724 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:40:45.050739 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:40:45.050774 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:40:45.053011 (+ 2237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:40:45.053578 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:40:45.053584 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:40:45.053589 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:40:45.053712 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:40:45.054050 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:40:45.054053 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:40:45.054055 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:40:45.054587 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:40:45.054773 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:45.055118 (+ 345us) server_negotiation.cc:299] Negotiation successful
0806 11:40:45.055443 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":140,"threads_started":1}
I20260806 11:40:45.059821 9673 raft_consensus.cc:348] T 00000000000000000000000000000000 P ed12a18ee96940db986f2da10542f0fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed12a18ee96940db986f2da10542f0fe" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 35875 } }
I20260806 11:40:45.060070 9673 raft_consensus.cc:374] T 00000000000000000000000000000000 P ed12a18ee96940db986f2da10542f0fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:40:45.060115 9673 raft_consensus.cc:729] T 00000000000000000000000000000000 P ed12a18ee96940db986f2da10542f0fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed12a18ee96940db986f2da10542f0fe, State: Initialized, Role: FOLLOWER
I20260806 11:40:45.060556 9673 consensus_queue.cc:260] T 00000000000000000000000000000000 P ed12a18ee96940db986f2da10542f0fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed12a18ee96940db986f2da10542f0fe" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 35875 } }
I20260806 11:40:45.060683 9673 raft_consensus.cc:388] T 00000000000000000000000000000000 P ed12a18ee96940db986f2da10542f0fe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 11:40:45.060732 9673 raft_consensus.cc:482] T 00000000000000000000000000000000 P ed12a18ee96940db986f2da10542f0fe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 11:40:45.060807 9673 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ed12a18ee96940db986f2da10542f0fe [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:40:45.061751 9673 raft_consensus.cc:504] T 00000000000000000000000000000000 P ed12a18ee96940db986f2da10542f0fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed12a18ee96940db986f2da10542f0fe" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 35875 } }
I20260806 11:40:45.062065 9673 leader_election.cc:302] T 00000000000000000000000000000000 P ed12a18ee96940db986f2da10542f0fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ed12a18ee96940db986f2da10542f0fe; no voters:
I20260806 11:40:45.062345 9673 leader_election.cc:288] T 00000000000000000000000000000000 P ed12a18ee96940db986f2da10542f0fe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 11:40:45.062502 9678 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ed12a18ee96940db986f2da10542f0fe [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:40:45.062775 9678 raft_consensus.cc:686] T 00000000000000000000000000000000 P ed12a18ee96940db986f2da10542f0fe [term 1 LEADER]: Becoming Leader. State: Replica: ed12a18ee96940db986f2da10542f0fe, State: Running, Role: LEADER
I20260806 11:40:45.063148 9678 consensus_queue.cc:237] T 00000000000000000000000000000000 P ed12a18ee96940db986f2da10542f0fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed12a18ee96940db986f2da10542f0fe" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 35875 } }
I20260806 11:40:45.063736 9673 sys_catalog.cc:564] T 00000000000000000000000000000000 P ed12a18ee96940db986f2da10542f0fe [sys.catalog]: configured and running, proceeding with master startup.
I20260806 11:40:45.064785 9680 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed12a18ee96940db986f2da10542f0fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader ed12a18ee96940db986f2da10542f0fe. Latest consensus state: current_term: 1 leader_uuid: "ed12a18ee96940db986f2da10542f0fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed12a18ee96940db986f2da10542f0fe" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 35875 } } }
I20260806 11:40:45.064916 9680 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed12a18ee96940db986f2da10542f0fe [sys.catalog]: This master's current role is: LEADER
I20260806 11:40:45.065296 9679 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed12a18ee96940db986f2da10542f0fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ed12a18ee96940db986f2da10542f0fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed12a18ee96940db986f2da10542f0fe" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 35875 } } }
I20260806 11:40:45.065407 9679 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed12a18ee96940db986f2da10542f0fe [sys.catalog]: This master's current role is: LEADER
I20260806 11:40:45.065445 9687 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 11:40:45.068849 9687 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 11:40:45.074769 9687 catalog_manager.cc:1347] Generated new cluster ID: a6a489f8b1154584917b1abe461d1aee
I20260806 11:40:45.074899 9687 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 11:40:45.086227 9687 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 11:40:45.087417 9687 catalog_manager.cc:1504] Loading token signing keys...
I20260806 11:40:45.105221 9687 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ed12a18ee96940db986f2da10542f0fe: Generated new TSK 0
I20260806 11:40:45.106096 9687 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.14.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab.
Entry for principal kudu/127.21.14.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab.
I20260806 11:40:45.142670 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.193:0
--local_ip_for_outbound_sockets=127.21.14.193
--webserver_interface=127.21.14.193
--webserver_port=0
--tserver_master_addrs=127.21.14.254:35875
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab
--principal=kudu/127.21.14.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:35921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:40:45.265621 9699 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:40:45.265969 9699 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:40:45.266050 9699 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:40:45.270404 9699 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:40:45.270524 9699 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:40:45.270624 9699 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.193
I20260806 11:40:45.276520 9699 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:35921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.193.keytab
--principal=kudu/127.21.14.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.14.193
--webserver_port=0
--tserver_master_addrs=127.21.14.254:35875
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9699
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.193
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:40:45.277818 9699 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:40:45.278964 9699 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:40:45.289816 9705 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:40:45.289812 9707 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:40:45.290000 9699 server_base.cc:1034] running on GCE node
W20260806 11:40:45.289812 9704 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:40:45.290750 9699 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:40:45.291529 9699 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:40:45.292735 9699 hybrid_clock.cc:648] HybridClock initialized: now 1786016445292694 us; error 54 us; skew 500 ppm
Aug 06 11:40:45 dist-test-slave-7c35 krb5kdc[9587](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016445, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:40:45.296010 9699 init.cc:377] Logged in from keytab as kudu/127.21.14.193@KRBTEST.COM (short username kudu)
I20260806 11:40:45.297019 9699 webserver.cc:460] Webserver started at http://127.21.14.193:42169/ using document root <none> and password file <none>
I20260806 11:40:45.297597 9699 fs_manager.cc:359] Metadata directory not provided
I20260806 11:40:45.297657 9699 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:40:45.297832 9699 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:40:45.300027 9699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/minicluster-data/ts-0/data/instance:
uuid: "3dd93576c53a4831b7c41c10bcaf866c"
format_stamp: "Formatted at 2026-08-06 11:40:45 on dist-test-slave-7c35"
server_key: "11b0b2e08d1d934a011581c19899b15f"
server_key_iv: "9eb871d4aa9002dc2f91a7624a41bad7"
server_key_version: "encryptionkey@0"
I20260806 11:40:45.300588 9699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/minicluster-data/ts-0/wal/instance:
uuid: "3dd93576c53a4831b7c41c10bcaf866c"
format_stamp: "Formatted at 2026-08-06 11:40:45 on dist-test-slave-7c35"
server_key: "11b0b2e08d1d934a011581c19899b15f"
server_key_iv: "9eb871d4aa9002dc2f91a7624a41bad7"
server_key_version: "encryptionkey@0"
I20260806 11:40:45.304719 9699 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 11:40:45.307601 9714 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:45.309144 9699 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 11:40:45.309326 9699 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/minicluster-data/ts-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/minicluster-data/ts-0/wal
uuid: "3dd93576c53a4831b7c41c10bcaf866c"
format_stamp: "Formatted at 2026-08-06 11:40:45 on dist-test-slave-7c35"
server_key: "11b0b2e08d1d934a011581c19899b15f"
server_key_iv: "9eb871d4aa9002dc2f91a7624a41bad7"
server_key_version: "encryptionkey@0"
I20260806 11:40:45.309478 9699 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:40:45.322712 9699 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:40:45.326666 9699 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:40:45.326965 9699 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:40:45.327682 9699 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:40:45.328660 9699 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:40:45.328740 9699 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:45.328817 9699 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:40:45.328862 9699 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:45.338946 9699 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.193:35921
I20260806 11:40:45.338989 9835 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.193:35921 every 8 connection(s)
I20260806 11:40:45.340094 9699 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
I20260806 11:40:45.341500 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 9699
I20260806 11:40:45.341652 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/minicluster-data/ts-0/wal/instance
I20260806 11:40:45.341984 21563 external_mini_cluster.cc:1428] Setting key 3b9a98caa737b9602b3fabebb2b39b75
Aug 06 11:40:45 dist-test-slave-7c35 krb5kdc[9587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016445, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.193@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.14.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.194@KRBTEST.COM" created.
I20260806 11:40:45.358083 9676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:45.342537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:37351 (local address 127.21.14.254:35875)
0806 11:40:45.342691 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:45.342696 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:45.343933 (+ 1237us) server_negotiation.cc:407] Connection header received
0806 11:40:45.345362 (+ 1429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:45.345365 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:45.345425 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:45.345510 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:40:45.347342 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:45.347932 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:45.349939 (+ 2007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:45.350212 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:45.350297 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:45.353517 (+ 3220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:40:45.353520 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:40:45.353522 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:40:45.353550 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:40:45.355320 (+ 1770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:40:45.355972 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:40:45.355976 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:40:45.355977 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:40:45.356155 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:40:45.356606 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:40:45.356612 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:40:45.356617 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:40:45.356863 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:40:45.357005 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:45.357788 (+ 783us) server_negotiation.cc:299] Negotiation successful
0806 11:40:45.357923 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 11:40:45.359472 9838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:45.343118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:35875 (local address 127.21.14.193:37351)
0806 11:40:45.343774 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:45.343823 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 11:40:45.345136 (+ 1313us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:40:45.345701 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:40:45.345709 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:40:45.346305 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:40:45.347139 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:45.347166 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:45.348097 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:45.348103 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:45.349780 (+ 1677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:45.349796 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:45.350403 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:45.350411 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:45.350696 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:45.351461 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:40:45.351496 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:40:45.353306 (+ 1810us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:40:45.355496 (+ 2190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:40:45.355511 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:40:45.355528 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:40:45.355824 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:40:45.356265 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:40:45.356268 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:40:45.356270 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:40:45.356448 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:40:45.357006 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:40:45.357017 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:40:45.357452 (+ 435us) client_negotiation.cc:769] Sending connection context
0806 11:40:45.357694 (+ 242us) client_negotiation.cc:240] Negotiation successful
0806 11:40:45.358130 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":183,"threads_started":1}
I20260806 11:40:45.361021 9836 heartbeater.cc:347] Connected to a master server at 127.21.14.254:35875
I20260806 11:40:45.361299 9836 heartbeater.cc:464] Registering TS with master...
I20260806 11:40:45.361871 9836 heartbeater.cc:511] Master 127.21.14.254:35875 requested a full tablet report, sending...
I20260806 11:40:45.363674 9637 ts_manager.cc:194] Registered new tserver with Master: 3dd93576c53a4831b7c41c10bcaf866c (127.21.14.193:35921)
I20260806 11:40:45.365013 9637 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.14.193@KRBTEST.COM'} at 127.21.14.193:37351
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab.
Entry for principal kudu/127.21.14.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab.
I20260806 11:40:45.380079 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.194:0
--local_ip_for_outbound_sockets=127.21.14.194
--webserver_interface=127.21.14.194
--webserver_port=0
--tserver_master_addrs=127.21.14.254:35875
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab
--principal=kudu/127.21.14.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:35921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:40:45.503250 9841 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:40:45.503566 9841 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:40:45.503614 9841 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:40:45.507874 9841 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:40:45.507947 9841 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:40:45.508013 9841 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.194
I20260806 11:40:45.513793 9841 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:35921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.194.keytab
--principal=kudu/127.21.14.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.14.194
--webserver_port=0
--tserver_master_addrs=127.21.14.254:35875
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9841
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.194
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:40:45.515198 9841 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:40:45.516294 9841 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:40:45.526660 9847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:40:45.526648 9846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:40:45.526647 9850 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:40:45.527931 9841 server_base.cc:1034] running on GCE node
I20260806 11:40:45.528537 9841 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:40:45.529417 9841 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:40:45.530601 9841 hybrid_clock.cc:648] HybridClock initialized: now 1786016445530566 us; error 56 us; skew 500 ppm
Aug 06 11:40:45 dist-test-slave-7c35 krb5kdc[9587](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016445, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:40:45.533638 9841 init.cc:377] Logged in from keytab as kudu/127.21.14.194@KRBTEST.COM (short username kudu)
I20260806 11:40:45.534632 9841 webserver.cc:460] Webserver started at http://127.21.14.194:38727/ using document root <none> and password file <none>
I20260806 11:40:45.535270 9841 fs_manager.cc:359] Metadata directory not provided
I20260806 11:40:45.535333 9841 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:40:45.535522 9841 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:40:45.537592 9841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/minicluster-data/ts-1/data/instance:
uuid: "5b9c51625c9342ebb7c6ec37a4028268"
format_stamp: "Formatted at 2026-08-06 11:40:45 on dist-test-slave-7c35"
server_key: "3c7fd06869e34922dfe70a949c91fba7"
server_key_iv: "6dc85d2213434c4b31740e1261fdd3a6"
server_key_version: "encryptionkey@0"
I20260806 11:40:45.538112 9841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/minicluster-data/ts-1/wal/instance:
uuid: "5b9c51625c9342ebb7c6ec37a4028268"
format_stamp: "Formatted at 2026-08-06 11:40:45 on dist-test-slave-7c35"
server_key: "3c7fd06869e34922dfe70a949c91fba7"
server_key_iv: "6dc85d2213434c4b31740e1261fdd3a6"
server_key_version: "encryptionkey@0"
I20260806 11:40:45.542114 9841 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 11:40:45.544895 9857 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:45.546465 9841 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 11:40:45.546631 9841 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/minicluster-data/ts-1/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/minicluster-data/ts-1/wal
uuid: "5b9c51625c9342ebb7c6ec37a4028268"
format_stamp: "Formatted at 2026-08-06 11:40:45 on dist-test-slave-7c35"
server_key: "3c7fd06869e34922dfe70a949c91fba7"
server_key_iv: "6dc85d2213434c4b31740e1261fdd3a6"
server_key_version: "encryptionkey@0"
I20260806 11:40:45.546769 9841 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:40:45.568789 9841 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:40:45.572726 9841 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:40:45.572969 9841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:40:45.573700 9841 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:40:45.574657 9841 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:40:45.574707 9841 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:45.574779 9841 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:40:45.574820 9841 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:45.584623 9841 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.194:45249
I20260806 11:40:45.584650 9976 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.194:45249 every 8 connection(s)
I20260806 11:40:45.585634 9841 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
I20260806 11:40:45.588725 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 9841
I20260806 11:40:45.588886 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/minicluster-data/ts-1/wal/instance
I20260806 11:40:45.589289 21563 external_mini_cluster.cc:1428] Setting key 1655fa4243c96308f5cd20beb6bbd18d
Aug 06 11:40:45 dist-test-slave-7c35 krb5kdc[9587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016445, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.194@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.14.195@KRBTEST.COM; defaulting to no policy
I20260806 11:40:45.603634 9676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:45.587956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:41191 (local address 127.21.14.254:35875)
0806 11:40:45.588147 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:45.588152 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:45.589893 (+ 1741us) server_negotiation.cc:407] Connection header received
0806 11:40:45.591087 (+ 1194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:45.591090 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:45.591148 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:45.591236 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:40:45.592781 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:45.593731 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:45.595223 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:45.595627 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:45.595765 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:45.598944 (+ 3179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:40:45.598951 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:40:45.598968 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:40:45.599016 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:40:45.600926 (+ 1910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:40:45.601656 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:40:45.601661 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:40:45.601663 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:40:45.601779 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:40:45.602176 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:40:45.602179 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:40:45.602180 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:40:45.602389 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:40:45.602471 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:45.603343 (+ 872us) server_negotiation.cc:299] Negotiation successful
0806 11:40:45.603491 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260806 11:40:45.604571 9979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:45.588341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:35875 (local address 127.21.14.194:41191)
0806 11:40:45.589673 (+ 1332us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:45.589744 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0806 11:40:45.590768 (+ 1024us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:40:45.591447 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:40:45.591455 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:40:45.591845 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:40:45.592579 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:45.592599 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:45.593916 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:45.593925 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:45.595022 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:45.595033 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:45.595878 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:45.595884 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:45.596077 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:45.596970 (+ 893us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:40:45.597017 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:40:45.598663 (+ 1646us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:40:45.601129 (+ 2466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:40:45.601136 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:40:45.601153 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:40:45.601488 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:40:45.601892 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:40:45.601894 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:40:45.601895 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:40:45.602080 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:40:45.602526 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:40:45.602533 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:40:45.603009 (+ 476us) client_negotiation.cc:769] Sending connection context
0806 11:40:45.603213 (+ 204us) client_negotiation.cc:240] Negotiation successful
0806 11:40:45.603535 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1080,"thread_start_us":801,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.195@KRBTEST.COM" created.
I20260806 11:40:45.606097 9977 heartbeater.cc:347] Connected to a master server at 127.21.14.254:35875
I20260806 11:40:45.606411 9977 heartbeater.cc:464] Registering TS with master...
I20260806 11:40:45.607197 9977 heartbeater.cc:511] Master 127.21.14.254:35875 requested a full tablet report, sending...
I20260806 11:40:45.608698 9637 ts_manager.cc:194] Registered new tserver with Master: 5b9c51625c9342ebb7c6ec37a4028268 (127.21.14.194:45249)
I20260806 11:40:45.609669 9637 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.14.194@KRBTEST.COM'} at 127.21.14.194:41191
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab.
Entry for principal kudu/127.21.14.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab.
I20260806 11:40:45.627570 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.195:0
--local_ip_for_outbound_sockets=127.21.14.195
--webserver_interface=127.21.14.195
--webserver_port=0
--tserver_master_addrs=127.21.14.254:35875
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab
--principal=kudu/127.21.14.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:35921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:40:45.746505 9982 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:40:45.746795 9982 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:40:45.746891 9982 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:40:45.751111 9982 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:40:45.751183 9982 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:40:45.751263 9982 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.195
I20260806 11:40:45.756928 9982 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:35921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.195.keytab
--principal=kudu/127.21.14.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.14.195
--webserver_port=0
--tserver_master_addrs=127.21.14.254:35875
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9982
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.195
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:40:45.758142 9982 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:40:45.759292 9982 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:40:45.769868 9988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:40:45.769815 9990 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:40:45.769892 9982 server_base.cc:1034] running on GCE node
W20260806 11:40:45.769845 9987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:40:45.770567 9982 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:40:45.771337 9982 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:40:45.772557 9982 hybrid_clock.cc:648] HybridClock initialized: now 1786016445772531 us; error 37 us; skew 500 ppm
Aug 06 11:40:45 dist-test-slave-7c35 krb5kdc[9587](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016445, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:40:45.775641 9982 init.cc:377] Logged in from keytab as kudu/127.21.14.195@KRBTEST.COM (short username kudu)
I20260806 11:40:45.776736 9982 webserver.cc:460] Webserver started at http://127.21.14.195:44375/ using document root <none> and password file <none>
I20260806 11:40:45.777402 9982 fs_manager.cc:359] Metadata directory not provided
I20260806 11:40:45.777509 9982 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:40:45.777783 9982 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:40:45.780226 9982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/minicluster-data/ts-2/data/instance:
uuid: "d5ab74c3d3564698825dba6b29cac70a"
format_stamp: "Formatted at 2026-08-06 11:40:45 on dist-test-slave-7c35"
server_key: "32276108e735fcaf643f09e2cce03237"
server_key_iv: "3f974092f22f9782f2a41a760f28ecef"
server_key_version: "encryptionkey@0"
I20260806 11:40:45.780978 9982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/minicluster-data/ts-2/wal/instance:
uuid: "d5ab74c3d3564698825dba6b29cac70a"
format_stamp: "Formatted at 2026-08-06 11:40:45 on dist-test-slave-7c35"
server_key: "32276108e735fcaf643f09e2cce03237"
server_key_iv: "3f974092f22f9782f2a41a760f28ecef"
server_key_version: "encryptionkey@0"
I20260806 11:40:45.785392 9982 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 11:40:45.788344 9997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:45.789851 9982 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 11:40:45.789973 9982 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/minicluster-data/ts-2/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/minicluster-data/ts-2/wal
uuid: "d5ab74c3d3564698825dba6b29cac70a"
format_stamp: "Formatted at 2026-08-06 11:40:45 on dist-test-slave-7c35"
server_key: "32276108e735fcaf643f09e2cce03237"
server_key_iv: "3f974092f22f9782f2a41a760f28ecef"
server_key_version: "encryptionkey@0"
I20260806 11:40:45.790078 9982 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:40:45.800905 9982 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:40:45.804428 9982 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:40:45.804608 9982 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:40:45.805186 9982 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:40:45.806012 9982 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:40:45.806058 9982 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:45.806098 9982 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:40:45.806123 9982 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:45.816128 9982 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.195:34307
I20260806 11:40:45.816154 10117 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.195:34307 every 8 connection(s)
I20260806 11:40:45.817116 9982 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
I20260806 11:40:45.826244 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 9982
I20260806 11:40:45.826462 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/minicluster-data/ts-2/wal/instance
I20260806 11:40:45.826787 21563 external_mini_cluster.cc:1428] Setting key 180d4b22cd1fd6854e1523c8e6ca181d
Aug 06 11:40:45 dist-test-slave-7c35 krb5kdc[9587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016445, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.195@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:40:45.832132 9676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:45.819158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:47619 (local address 127.21.14.254:35875)
0806 11:40:45.819310 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:45.819315 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:45.820287 (+ 972us) server_negotiation.cc:407] Connection header received
0806 11:40:45.821706 (+ 1419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:45.821710 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:45.821777 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:45.821890 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:40:45.823241 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:45.823795 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:45.825034 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:45.825262 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:45.825370 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:45.828242 (+ 2872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:40:45.828246 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:40:45.828249 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:40:45.828281 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:40:45.829892 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:40:45.830436 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:40:45.830441 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:40:45.830442 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:40:45.830526 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:40:45.830945 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:40:45.830948 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:40:45.830950 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:40:45.831145 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:40:45.831257 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:45.831818 (+ 561us) server_negotiation.cc:299] Negotiation successful
0806 11:40:45.831957 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 11:40:45.832959 10120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:45.819591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:35875 (local address 127.21.14.195:47619)
0806 11:40:45.820099 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:45.820154 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 11:40:45.821463 (+ 1309us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:40:45.822070 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:40:45.822078 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:40:45.822469 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:40:45.823068 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:45.823085 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:45.823945 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:45.823948 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:45.824892 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:45.824901 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:45.825472 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:45.825476 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:45.825637 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:45.826290 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:40:45.826308 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:40:45.828068 (+ 1760us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:40:45.830011 (+ 1943us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:40:45.830018 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:40:45.830035 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:40:45.830320 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:40:45.830623 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:40:45.830626 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:40:45.830628 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:40:45.830800 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:40:45.831257 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:40:45.831266 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:40:45.831568 (+ 302us) client_negotiation.cc:769] Sending connection context
0806 11:40:45.831760 (+ 192us) client_negotiation.cc:240] Negotiation successful
0806 11:40:45.832106 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":128,"threads_started":1}
I20260806 11:40:45.834172 10118 heartbeater.cc:347] Connected to a master server at 127.21.14.254:35875
I20260806 11:40:45.834441 10118 heartbeater.cc:464] Registering TS with master...
I20260806 11:40:45.835006 10118 heartbeater.cc:511] Master 127.21.14.254:35875 requested a full tablet report, sending...
I20260806 11:40:45.836138 9637 ts_manager.cc:194] Registered new tserver with Master: d5ab74c3d3564698825dba6b29cac70a (127.21.14.195:34307)
I20260806 11:40:45.836834 9637 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.14.195@KRBTEST.COM'} at 127.21.14.195:47619
I20260806 11:40:45.841523 21563 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 11:40:45.852638 9676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:45.843891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49472 (local address 127.21.14.254:35875)
0806 11:40:45.844042 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:45.844046 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:45.844273 (+ 227us) server_negotiation.cc:407] Connection header received
0806 11:40:45.844380 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:45.844383 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:45.844432 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:45.844507 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:40:45.845369 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:45.845916 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:45.846840 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:45.847170 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:45.847263 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:45.848709 (+ 1446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:40:45.848713 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:40:45.848715 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:40:45.848741 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:40:45.850170 (+ 1429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:40:45.851244 (+ 1074us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:40:45.851247 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:40:45.851248 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:40:45.851334 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:40:45.851610 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:40:45.851613 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:40:45.851614 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:40:45.851949 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:40:45.852075 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:45.852391 (+ 316us) server_negotiation.cc:299] Negotiation successful
0806 11:40:45.852502 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 11:40:45.855948 9637 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49472:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 11:40:45.859233 9637 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 11:40:45.875458 10130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:45.869155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44094 (local address 127.21.14.194:45249)
0806 11:40:45.869985 (+ 830us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:45.869999 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:45.870604 (+ 605us) server_negotiation.cc:407] Connection header received
0806 11:40:45.870780 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:45.870785 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:45.871007 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:45.871148 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:40:45.871647 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:45.872446 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:45.874021 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:45.874690 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:45.874804 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:45.875006 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:45.875192 (+ 186us) server_negotiation.cc:299] Negotiation successful
0806 11:40:45.875303 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":715,"thread_start_us":145,"threads_started":1}
I20260806 11:40:45.875447 10129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:45.869018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.194:45249 (local address 127.0.0.1:44094)
0806 11:40:45.870494 (+ 1476us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:45.870508 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 11:40:45.870633 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:40:45.871366 (+ 733us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:40:45.871369 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:40:45.871384 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:40:45.871511 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:45.871517 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:45.872619 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:45.872622 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:45.873876 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:45.873886 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:45.874826 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:45.874830 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:45.874975 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:45.875022 (+ 47us) client_negotiation.cc:769] Sending connection context
0806 11:40:45.875142 (+ 120us) client_negotiation.cc:240] Negotiation successful
0806 11:40:45.875269 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1377,"mutex_wait_us":304,"thread_start_us":115,"threads_started":1}
I20260806 11:40:45.876351 10132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:45.869018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.195:34307 (local address 127.0.0.1:36884)
0806 11:40:45.870298 (+ 1280us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:45.870351 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 11:40:45.870518 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:40:45.871190 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:40:45.871196 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:40:45.871219 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:40:45.871379 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:45.871386 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:45.872932 (+ 1546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:45.872937 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:45.874501 (+ 1564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:45.874513 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:45.875917 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:45.875921 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:45.876019 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:45.876046 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 11:40:45.876126 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 11:40:45.876206 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1094,"mutex_wait_us":123,"thread_start_us":113,"threads_started":1}
I20260806 11:40:45.876422 10134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:45.869934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51326 (local address 127.21.14.193:35921)
0806 11:40:45.870365 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:45.870375 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:45.870613 (+ 238us) server_negotiation.cc:407] Connection header received
0806 11:40:45.870811 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:45.870815 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:45.871035 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:45.871194 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:40:45.871709 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:45.872832 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:45.874651 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:45.875533 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:45.875650 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:45.875796 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:45.876170 (+ 374us) server_negotiation.cc:299] Negotiation successful
0806 11:40:45.876277 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":159,"threads_started":1}
I20260806 11:40:45.876423 10133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:45.869818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36884 (local address 127.21.14.195:34307)
0806 11:40:45.870312 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:45.870326 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:45.870462 (+ 136us) server_negotiation.cc:407] Connection header received
0806 11:40:45.870744 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:45.870749 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:45.871006 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:45.871158 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:40:45.871502 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:45.872758 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:45.874663 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:45.875594 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:45.875709 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:45.875857 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:45.876180 (+ 323us) server_negotiation.cc:299] Negotiation successful
0806 11:40:45.876277 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":111,"threads_started":1}
I20260806 11:40:45.876796 10131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:45.869299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.193:35921 (local address 127.0.0.1:51326)
0806 11:40:45.870516 (+ 1217us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:45.870528 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 11:40:45.870685 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:40:45.871481 (+ 796us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:40:45.871483 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:40:45.871498 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:40:45.871560 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:45.871563 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:45.872950 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:45.872953 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:45.874501 (+ 1548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:45.874513 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:45.875686 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:45.875690 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:45.875981 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:45.876019 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 11:40:45.876621 (+ 602us) client_negotiation.cc:240] Negotiation successful
0806 11:40:45.876698 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1160,"mutex_wait_us":273,"thread_start_us":93,"threads_started":1}
I20260806 11:40:45.879590 9766 tablet_service.cc:1467] Processing CreateTablet for tablet 74fdbf42cecb46e3b8a08eb6b1815928 (DEFAULT_TABLE table=test-table [id=76541767f8b44e90a48bd82d0f657f14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:40:45.879590 10045 tablet_service.cc:1467] Processing CreateTablet for tablet 74fdbf42cecb46e3b8a08eb6b1815928 (DEFAULT_TABLE table=test-table [id=76541767f8b44e90a48bd82d0f657f14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:40:45.880808 10045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74fdbf42cecb46e3b8a08eb6b1815928. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:40:45.880808 9766 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74fdbf42cecb46e3b8a08eb6b1815928. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:40:45.880609 9907 tablet_service.cc:1467] Processing CreateTablet for tablet 74fdbf42cecb46e3b8a08eb6b1815928 (DEFAULT_TABLE table=test-table [id=76541767f8b44e90a48bd82d0f657f14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:40:45.881574 9907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74fdbf42cecb46e3b8a08eb6b1815928. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:40:45.888055 10135 tablet_bootstrap.cc:492] T 74fdbf42cecb46e3b8a08eb6b1815928 P 5b9c51625c9342ebb7c6ec37a4028268: Bootstrap starting.
I20260806 11:40:45.889402 10136 tablet_bootstrap.cc:492] T 74fdbf42cecb46e3b8a08eb6b1815928 P 3dd93576c53a4831b7c41c10bcaf866c: Bootstrap starting.
I20260806 11:40:45.889549 10137 tablet_bootstrap.cc:492] T 74fdbf42cecb46e3b8a08eb6b1815928 P d5ab74c3d3564698825dba6b29cac70a: Bootstrap starting.
I20260806 11:40:45.891222 10135 tablet_bootstrap.cc:654] T 74fdbf42cecb46e3b8a08eb6b1815928 P 5b9c51625c9342ebb7c6ec37a4028268: Neither blocks nor log segments found. Creating new log.
I20260806 11:40:45.891660 10136 tablet_bootstrap.cc:654] T 74fdbf42cecb46e3b8a08eb6b1815928 P 3dd93576c53a4831b7c41c10bcaf866c: Neither blocks nor log segments found. Creating new log.
I20260806 11:40:45.891685 10137 tablet_bootstrap.cc:654] T 74fdbf42cecb46e3b8a08eb6b1815928 P d5ab74c3d3564698825dba6b29cac70a: Neither blocks nor log segments found. Creating new log.
I20260806 11:40:45.892653 10135 log.cc:824] T 74fdbf42cecb46e3b8a08eb6b1815928 P 5b9c51625c9342ebb7c6ec37a4028268: Log is configured to *not* fsync() on all Append() calls
I20260806 11:40:45.892654 10136 log.cc:824] T 74fdbf42cecb46e3b8a08eb6b1815928 P 3dd93576c53a4831b7c41c10bcaf866c: Log is configured to *not* fsync() on all Append() calls
I20260806 11:40:45.892709 10137 log.cc:824] T 74fdbf42cecb46e3b8a08eb6b1815928 P d5ab74c3d3564698825dba6b29cac70a: Log is configured to *not* fsync() on all Append() calls
I20260806 11:40:45.894785 10135 tablet_bootstrap.cc:492] T 74fdbf42cecb46e3b8a08eb6b1815928 P 5b9c51625c9342ebb7c6ec37a4028268: No bootstrap required, opened a new log
I20260806 11:40:45.894882 10136 tablet_bootstrap.cc:492] T 74fdbf42cecb46e3b8a08eb6b1815928 P 3dd93576c53a4831b7c41c10bcaf866c: No bootstrap required, opened a new log
I20260806 11:40:45.894882 10137 tablet_bootstrap.cc:492] T 74fdbf42cecb46e3b8a08eb6b1815928 P d5ab74c3d3564698825dba6b29cac70a: No bootstrap required, opened a new log
I20260806 11:40:45.895088 10136 ts_tablet_manager.cc:1397] T 74fdbf42cecb46e3b8a08eb6b1815928 P 3dd93576c53a4831b7c41c10bcaf866c: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 11:40:45.895076 10135 ts_tablet_manager.cc:1397] T 74fdbf42cecb46e3b8a08eb6b1815928 P 5b9c51625c9342ebb7c6ec37a4028268: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260806 11:40:45.895088 10137 ts_tablet_manager.cc:1397] T 74fdbf42cecb46e3b8a08eb6b1815928 P d5ab74c3d3564698825dba6b29cac70a: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 11:40:45.898595 10135 raft_consensus.cc:348] T 74fdbf42cecb46e3b8a08eb6b1815928 P 5b9c51625c9342ebb7c6ec37a4028268 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9c51625c9342ebb7c6ec37a4028268" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 45249 } } peers { permanent_uuid: "d5ab74c3d3564698825dba6b29cac70a" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 34307 } } peers { permanent_uuid: "3dd93576c53a4831b7c41c10bcaf866c" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 35921 } }
I20260806 11:40:45.898854 10135 raft_consensus.cc:374] T 74fdbf42cecb46e3b8a08eb6b1815928 P 5b9c51625c9342ebb7c6ec37a4028268 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:40:45.898931 10135 raft_consensus.cc:729] T 74fdbf42cecb46e3b8a08eb6b1815928 P 5b9c51625c9342ebb7c6ec37a4028268 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b9c51625c9342ebb7c6ec37a4028268, State: Initialized, Role: FOLLOWER
I20260806 11:40:45.899397 10135 consensus_queue.cc:260] T 74fdbf42cecb46e3b8a08eb6b1815928 P 5b9c51625c9342ebb7c6ec37a4028268 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9c51625c9342ebb7c6ec37a4028268" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 45249 } } peers { permanent_uuid: "d5ab74c3d3564698825dba6b29cac70a" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 34307 } } peers { permanent_uuid: "3dd93576c53a4831b7c41c10bcaf866c" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 35921 } }
I20260806 11:40:45.899521 10137 raft_consensus.cc:348] T 74fdbf42cecb46e3b8a08eb6b1815928 P d5ab74c3d3564698825dba6b29cac70a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9c51625c9342ebb7c6ec37a4028268" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 45249 } } peers { permanent_uuid: "d5ab74c3d3564698825dba6b29cac70a" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 34307 } } peers { permanent_uuid: "3dd93576c53a4831b7c41c10bcaf866c" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 35921 } }
I20260806 11:40:45.899525 10136 raft_consensus.cc:348] T 74fdbf42cecb46e3b8a08eb6b1815928 P 3dd93576c53a4831b7c41c10bcaf866c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9c51625c9342ebb7c6ec37a4028268" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 45249 } } peers { permanent_uuid: "d5ab74c3d3564698825dba6b29cac70a" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 34307 } } peers { permanent_uuid: "3dd93576c53a4831b7c41c10bcaf866c" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 35921 } }
I20260806 11:40:45.899791 10137 raft_consensus.cc:374] T 74fdbf42cecb46e3b8a08eb6b1815928 P d5ab74c3d3564698825dba6b29cac70a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:40:45.899791 10136 raft_consensus.cc:374] T 74fdbf42cecb46e3b8a08eb6b1815928 P 3dd93576c53a4831b7c41c10bcaf866c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:40:45.899852 10137 raft_consensus.cc:729] T 74fdbf42cecb46e3b8a08eb6b1815928 P d5ab74c3d3564698825dba6b29cac70a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5ab74c3d3564698825dba6b29cac70a, State: Initialized, Role: FOLLOWER
I20260806 11:40:45.899852 10136 raft_consensus.cc:729] T 74fdbf42cecb46e3b8a08eb6b1815928 P 3dd93576c53a4831b7c41c10bcaf866c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3dd93576c53a4831b7c41c10bcaf866c, State: Initialized, Role: FOLLOWER
I20260806 11:40:45.900297 10137 consensus_queue.cc:260] T 74fdbf42cecb46e3b8a08eb6b1815928 P d5ab74c3d3564698825dba6b29cac70a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9c51625c9342ebb7c6ec37a4028268" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 45249 } } peers { permanent_uuid: "d5ab74c3d3564698825dba6b29cac70a" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 34307 } } peers { permanent_uuid: "3dd93576c53a4831b7c41c10bcaf866c" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 35921 } }
I20260806 11:40:45.900298 10136 consensus_queue.cc:260] T 74fdbf42cecb46e3b8a08eb6b1815928 P 3dd93576c53a4831b7c41c10bcaf866c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9c51625c9342ebb7c6ec37a4028268" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 45249 } } peers { permanent_uuid: "d5ab74c3d3564698825dba6b29cac70a" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 34307 } } peers { permanent_uuid: "3dd93576c53a4831b7c41c10bcaf866c" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 35921 } }
I20260806 11:40:45.900377 9977 heartbeater.cc:503] Master 127.21.14.254:35875 was elected leader, sending a full tablet report...
I20260806 11:40:45.900543 10135 ts_tablet_manager.cc:1428] T 74fdbf42cecb46e3b8a08eb6b1815928 P 5b9c51625c9342ebb7c6ec37a4028268: Time spent starting tablet: real 0.005s user 0.001s sys 0.004s
I20260806 11:40:45.901276 10136 ts_tablet_manager.cc:1428] T 74fdbf42cecb46e3b8a08eb6b1815928 P 3dd93576c53a4831b7c41c10bcaf866c: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 11:40:45.901276 10137 ts_tablet_manager.cc:1428] T 74fdbf42cecb46e3b8a08eb6b1815928 P d5ab74c3d3564698825dba6b29cac70a: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260806 11:40:45.901659 9836 heartbeater.cc:503] Master 127.21.14.254:35875 was elected leader, sending a full tablet report...
I20260806 11:40:45.901844 10118 heartbeater.cc:503] Master 127.21.14.254:35875 was elected leader, sending a full tablet report...
I20260806 11:40:45.906708 10141 raft_consensus.cc:482] T 74fdbf42cecb46e3b8a08eb6b1815928 P 5b9c51625c9342ebb7c6ec37a4028268 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 11:40:45.906957 10141 raft_consensus.cc:504] T 74fdbf42cecb46e3b8a08eb6b1815928 P 5b9c51625c9342ebb7c6ec37a4028268 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9c51625c9342ebb7c6ec37a4028268" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 45249 } } peers { permanent_uuid: "d5ab74c3d3564698825dba6b29cac70a" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 34307 } } peers { permanent_uuid: "3dd93576c53a4831b7c41c10bcaf866c" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 35921 } }
I20260806 11:40:45.908147 10141 leader_election.cc:288] T 74fdbf42cecb46e3b8a08eb6b1815928 P 5b9c51625c9342ebb7c6ec37a4028268 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d5ab74c3d3564698825dba6b29cac70a (127.21.14.195:34307), 3dd93576c53a4831b7c41c10bcaf866c (127.21.14.193:35921)
I20260806 11:40:45.912287 10134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:45.908658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:58205 (local address 127.21.14.193:35921)
0806 11:40:45.908785 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:45.908790 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:45.908987 (+ 197us) server_negotiation.cc:407] Connection header received
0806 11:40:45.909132 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:45.909136 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:45.909187 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:45.909273 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:40:45.909556 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:45.910060 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:45.911212 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:45.911649 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:45.911782 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:45.911842 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:45.912037 (+ 195us) server_negotiation.cc:299] Negotiation successful
0806 11:40:45.912142 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 11:40:45.912257 10144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:45.908380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.193:35921 (local address 127.21.14.194:58205)
0806 11:40:45.908885 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:45.908903 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 11:40:45.909019 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:40:45.909286 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:40:45.909289 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:40:45.909305 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:40:45.909439 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:45.909444 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:45.910193 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:45.910197 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:45.911081 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:45.911091 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:45.911778 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:45.911782 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:45.911886 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:45.911913 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 11:40:45.911998 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 11:40:45.912088 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"mutex_wait_us":97,"thread_start_us":124,"threads_started":1}
I20260806 11:40:45.912608 10133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:45.908521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:54479 (local address 127.21.14.195:34307)
0806 11:40:45.908641 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:45.908646 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:45.908663 (+ 17us) server_negotiation.cc:407] Connection header received
0806 11:40:45.908730 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:45.908733 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:45.908784 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:45.908874 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:40:45.909177 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:45.909831 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:45.911196 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:45.911642 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:45.911789 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:45.911840 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:45.912390 (+ 550us) server_negotiation.cc:299] Negotiation successful
0806 11:40:45.912488 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 11:40:45.912922 9979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:45.908367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.195:34307 (local address 127.21.14.194:54479)
0806 11:40:45.908499 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:45.908515 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 11:40:45.908645 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:40:45.908887 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:40:45.908891 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:40:45.908914 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:40:45.909042 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:45.909049 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:45.909951 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:45.909956 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:45.911060 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:45.911071 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:45.911911 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:45.911915 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:45.912066 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:45.912091 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 11:40:45.912167 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 11:40:45.912244 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260806 11:40:45.912971 9787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74fdbf42cecb46e3b8a08eb6b1815928" candidate_uuid: "5b9c51625c9342ebb7c6ec37a4028268" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3dd93576c53a4831b7c41c10bcaf866c" is_pre_election: true
I20260806 11:40:45.913341 9787 raft_consensus.cc:2454] T 74fdbf42cecb46e3b8a08eb6b1815928 P 3dd93576c53a4831b7c41c10bcaf866c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b9c51625c9342ebb7c6ec37a4028268 in term 0.
I20260806 11:40:45.913666 10067 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74fdbf42cecb46e3b8a08eb6b1815928" candidate_uuid: "5b9c51625c9342ebb7c6ec37a4028268" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5ab74c3d3564698825dba6b29cac70a" is_pre_election: true
I20260806 11:40:45.913928 9861 leader_election.cc:302] T 74fdbf42cecb46e3b8a08eb6b1815928 P 5b9c51625c9342ebb7c6ec37a4028268 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3dd93576c53a4831b7c41c10bcaf866c, 5b9c51625c9342ebb7c6ec37a4028268; no voters:
I20260806 11:40:45.914000 10067 raft_consensus.cc:2454] T 74fdbf42cecb46e3b8a08eb6b1815928 P d5ab74c3d3564698825dba6b29cac70a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b9c51625c9342ebb7c6ec37a4028268 in term 0.
I20260806 11:40:45.914279 10141 raft_consensus.cc:2781] T 74fdbf42cecb46e3b8a08eb6b1815928 P 5b9c51625c9342ebb7c6ec37a4028268 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 11:40:45.914387 10141 raft_consensus.cc:482] T 74fdbf42cecb46e3b8a08eb6b1815928 P 5b9c51625c9342ebb7c6ec37a4028268 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 11:40:45.914449 10141 raft_consensus.cc:3037] T 74fdbf42cecb46e3b8a08eb6b1815928 P 5b9c51625c9342ebb7c6ec37a4028268 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:40:45.915702 10141 raft_consensus.cc:504] T 74fdbf42cecb46e3b8a08eb6b1815928 P 5b9c51625c9342ebb7c6ec37a4028268 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9c51625c9342ebb7c6ec37a4028268" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 45249 } } peers { permanent_uuid: "d5ab74c3d3564698825dba6b29cac70a" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 34307 } } peers { permanent_uuid: "3dd93576c53a4831b7c41c10bcaf866c" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 35921 } }
I20260806 11:40:45.916131 10141 leader_election.cc:288] T 74fdbf42cecb46e3b8a08eb6b1815928 P 5b9c51625c9342ebb7c6ec37a4028268 [CANDIDATE]: Term 1 election: Requested vote from peers d5ab74c3d3564698825dba6b29cac70a (127.21.14.195:34307), 3dd93576c53a4831b7c41c10bcaf866c (127.21.14.193:35921)
I20260806 11:40:45.916533 10067 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74fdbf42cecb46e3b8a08eb6b1815928" candidate_uuid: "5b9c51625c9342ebb7c6ec37a4028268" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5ab74c3d3564698825dba6b29cac70a"
I20260806 11:40:45.916576 9787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74fdbf42cecb46e3b8a08eb6b1815928" candidate_uuid: "5b9c51625c9342ebb7c6ec37a4028268" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3dd93576c53a4831b7c41c10bcaf866c"
I20260806 11:40:45.916679 10067 raft_consensus.cc:3037] T 74fdbf42cecb46e3b8a08eb6b1815928 P d5ab74c3d3564698825dba6b29cac70a [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:40:45.916700 9787 raft_consensus.cc:3037] T 74fdbf42cecb46e3b8a08eb6b1815928 P 3dd93576c53a4831b7c41c10bcaf866c [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:40:45.918130 10067 raft_consensus.cc:2454] T 74fdbf42cecb46e3b8a08eb6b1815928 P d5ab74c3d3564698825dba6b29cac70a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b9c51625c9342ebb7c6ec37a4028268 in term 1.
I20260806 11:40:45.918130 9787 raft_consensus.cc:2454] T 74fdbf42cecb46e3b8a08eb6b1815928 P 3dd93576c53a4831b7c41c10bcaf866c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b9c51625c9342ebb7c6ec37a4028268 in term 1.
I20260806 11:40:45.918504 9861 leader_election.cc:302] T 74fdbf42cecb46e3b8a08eb6b1815928 P 5b9c51625c9342ebb7c6ec37a4028268 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3dd93576c53a4831b7c41c10bcaf866c, 5b9c51625c9342ebb7c6ec37a4028268; no voters:
I20260806 11:40:45.918799 10141 raft_consensus.cc:2781] T 74fdbf42cecb46e3b8a08eb6b1815928 P 5b9c51625c9342ebb7c6ec37a4028268 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:40:45.919070 10141 raft_consensus.cc:686] T 74fdbf42cecb46e3b8a08eb6b1815928 P 5b9c51625c9342ebb7c6ec37a4028268 [term 1 LEADER]: Becoming Leader. State: Replica: 5b9c51625c9342ebb7c6ec37a4028268, State: Running, Role: LEADER
I20260806 11:40:45.919412 10141 consensus_queue.cc:237] T 74fdbf42cecb46e3b8a08eb6b1815928 P 5b9c51625c9342ebb7c6ec37a4028268 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9c51625c9342ebb7c6ec37a4028268" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 45249 } } peers { permanent_uuid: "d5ab74c3d3564698825dba6b29cac70a" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 34307 } } peers { permanent_uuid: "3dd93576c53a4831b7c41c10bcaf866c" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 35921 } }
I20260806 11:40:45.922798 9636 catalog_manager.cc:5495] T 74fdbf42cecb46e3b8a08eb6b1815928 P 5b9c51625c9342ebb7c6ec37a4028268 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5b9c51625c9342ebb7c6ec37a4028268 (127.21.14.194). New cstate: current_term: 1 leader_uuid: "5b9c51625c9342ebb7c6ec37a4028268" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9c51625c9342ebb7c6ec37a4028268" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 45249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d5ab74c3d3564698825dba6b29cac70a" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 34307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3dd93576c53a4831b7c41c10bcaf866c" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 35921 } health_report { overall_health: UNKNOWN } } }
I20260806 11:40:45.945675 10130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:45.941892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44110 (local address 127.21.14.194:45249)
0806 11:40:45.942064 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:45.942069 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:45.942084 (+ 15us) server_negotiation.cc:407] Connection header received
0806 11:40:45.942194 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:45.942197 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:45.942256 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:45.942378 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 11:40:45.942649 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:45.943262 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:45.944128 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:45.944343 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:45.944452 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:45.944795 (+ 343us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 11:40:45.945251 (+ 456us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 11:40:45.945391 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:45.945433 (+ 42us) server_negotiation.cc:299] Negotiation successful
0806 11:40:45.945495 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 11:40:45.952204 10067 raft_consensus.cc:1261] T 74fdbf42cecb46e3b8a08eb6b1815928 P d5ab74c3d3564698825dba6b29cac70a [term 1 FOLLOWER]: Refusing update from remote peer 5b9c51625c9342ebb7c6ec37a4028268: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 11:40:45.952345 9787 raft_consensus.cc:1261] T 74fdbf42cecb46e3b8a08eb6b1815928 P 3dd93576c53a4831b7c41c10bcaf866c [term 1 FOLLOWER]: Refusing update from remote peer 5b9c51625c9342ebb7c6ec37a4028268: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 11:40:45.953091 10145 consensus_queue.cc:1035] T 74fdbf42cecb46e3b8a08eb6b1815928 P 5b9c51625c9342ebb7c6ec37a4028268 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5ab74c3d3564698825dba6b29cac70a" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 34307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 11:40:45.953310 10141 consensus_queue.cc:1035] T 74fdbf42cecb46e3b8a08eb6b1815928 P 5b9c51625c9342ebb7c6ec37a4028268 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3dd93576c53a4831b7c41c10bcaf866c" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 35921 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 11:40:45.962311 10148 mvcc.cc:205] Tried to move back new op lower bound from 7315523362611052544 to 7315523362488426496. Current Snapshot: MvccSnapshot[applied={T|T < 7315523362611052544}]
I20260806 11:40:45.972230 9636 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49472:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 11:40:45.972400 9636 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49472:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 11:40:45.975194 9636 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ed12a18ee96940db986f2da10542f0fe: Sending DeleteTablet for 3 replicas of tablet 74fdbf42cecb46e3b8a08eb6b1815928
I20260806 11:40:45.976104 9907 tablet_service.cc:1514] Processing DeleteTablet for tablet 74fdbf42cecb46e3b8a08eb6b1815928 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:40:45 UTC) from {username='kudu', principal='kudu/127.21.14.254@KRBTEST.COM'} at 127.0.0.1:44094
I20260806 11:40:45.976174 10045 tablet_service.cc:1514] Processing DeleteTablet for tablet 74fdbf42cecb46e3b8a08eb6b1815928 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:40:45 UTC) from {username='kudu', principal='kudu/127.21.14.254@KRBTEST.COM'} at 127.0.0.1:36884
I20260806 11:40:45.976239 9766 tablet_service.cc:1514] Processing DeleteTablet for tablet 74fdbf42cecb46e3b8a08eb6b1815928 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:40:45 UTC) from {username='kudu', principal='kudu/127.21.14.254@KRBTEST.COM'} at 127.0.0.1:51326
I20260806 11:40:45.976766 10159 tablet_replica.cc:330] T 74fdbf42cecb46e3b8a08eb6b1815928 P d5ab74c3d3564698825dba6b29cac70a: stopping tablet replica
I20260806 11:40:45.977154 10159 raft_consensus.cc:2229] T 74fdbf42cecb46e3b8a08eb6b1815928 P d5ab74c3d3564698825dba6b29cac70a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 11:40:45.977221 10157 tablet_replica.cc:330] T 74fdbf42cecb46e3b8a08eb6b1815928 P 5b9c51625c9342ebb7c6ec37a4028268: stopping tablet replica
I20260806 11:40:45.977509 10157 raft_consensus.cc:2229] T 74fdbf42cecb46e3b8a08eb6b1815928 P 5b9c51625c9342ebb7c6ec37a4028268 [term 1 LEADER]: Raft consensus shutting down.
I20260806 11:40:45.977701 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 9699
I20260806 11:40:45.983529 10157 raft_consensus.cc:2258] T 74fdbf42cecb46e3b8a08eb6b1815928 P 5b9c51625c9342ebb7c6ec37a4028268 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 11:40:45.984570 10159 raft_consensus.cc:2258] T 74fdbf42cecb46e3b8a08eb6b1815928 P d5ab74c3d3564698825dba6b29cac70a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 11:40:45.985569 10157 ts_tablet_manager.cc:1905] T 74fdbf42cecb46e3b8a08eb6b1815928 P 5b9c51625c9342ebb7c6ec37a4028268: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 11:40:45.986202 10159 ts_tablet_manager.cc:1905] T 74fdbf42cecb46e3b8a08eb6b1815928 P d5ab74c3d3564698825dba6b29cac70a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 11:40:45.988903 10157 ts_tablet_manager.cc:1918] T 74fdbf42cecb46e3b8a08eb6b1815928 P 5b9c51625c9342ebb7c6ec37a4028268: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 11:40:45.989056 10157 log.cc:1196] T 74fdbf42cecb46e3b8a08eb6b1815928 P 5b9c51625c9342ebb7c6ec37a4028268: Deleting WAL directory at /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/minicluster-data/ts-1/wal/wals/74fdbf42cecb46e3b8a08eb6b1815928
I20260806 11:40:45.989392 10159 ts_tablet_manager.cc:1918] T 74fdbf42cecb46e3b8a08eb6b1815928 P d5ab74c3d3564698825dba6b29cac70a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 11:40:45.989454 10157 ts_tablet_manager.cc:1939] T 74fdbf42cecb46e3b8a08eb6b1815928 P 5b9c51625c9342ebb7c6ec37a4028268: Deleting consensus metadata
I20260806 11:40:45.989518 10159 log.cc:1196] T 74fdbf42cecb46e3b8a08eb6b1815928 P d5ab74c3d3564698825dba6b29cac70a: Deleting WAL directory at /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786016169045816-21563-0/minicluster-data/ts-2/wal/wals/74fdbf42cecb46e3b8a08eb6b1815928
I20260806 11:40:45.989856 10159 ts_tablet_manager.cc:1939] T 74fdbf42cecb46e3b8a08eb6b1815928 P d5ab74c3d3564698825dba6b29cac70a: Deleting consensus metadata
I20260806 11:40:45.991004 9622 catalog_manager.cc:4841] TS 5b9c51625c9342ebb7c6ec37a4028268 (127.21.14.194:45249): tablet 74fdbf42cecb46e3b8a08eb6b1815928 (table test-table [id=76541767f8b44e90a48bd82d0f657f14]) successfully deleted
I20260806 11:40:45.991004 9623 catalog_manager.cc:4841] TS d5ab74c3d3564698825dba6b29cac70a (127.21.14.195:34307): tablet 74fdbf42cecb46e3b8a08eb6b1815928 (table test-table [id=76541767f8b44e90a48bd82d0f657f14]) successfully deleted
W20260806 11:40:45.992803 9624 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.14.193:35921 (error 108)
I20260806 11:40:45.993824 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 9841
I20260806 11:40:45.994194 10129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:45.993867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.193:35921 (local address 127.0.0.1:51340)
0806 11:40:45.994020 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:45.994103 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.193:35921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260806 11:40:45.996263 9624 catalog_manager.cc:4584] TS 3dd93576c53a4831b7c41c10bcaf866c (127.21.14.193:35921): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 74fdbf42cecb46e3b8a08eb6b1815928: Network error: Client connection negotiation failed: client connection to 127.21.14.193:35921: connect: Connection refused (error 111)
I20260806 11:40:46.002940 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 9982
I20260806 11:40:46.010427 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 9604
2026-08-06T11:40:46Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2425 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-06T11:40:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T11:40:46Z Disabled control of system clock
I20260806 11:40:46.045773 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786016169045816-21563-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786016169045816-21563-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786016169045816-21563-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:36091
--webserver_interface=127.21.14.254
--webserver_port=0
--builtin_ntp_servers=127.21.14.212:33153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.14.254:36091
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 11:40:46.165920 10164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:40:46.166234 10164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:40:46.166271 10164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:40:46.170385 10164 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 11:40:46.170459 10164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:40:46.170475 10164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:40:46.170492 10164 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 11:40:46.170507 10164 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 11:40:46.175964 10164 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:33153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786016169045816-21563-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786016169045816-21563-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.14.254:36091
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:36091
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.14.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10164
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786016169045816-21563-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:40:46.177229 10164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:40:46.179013 10164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 11:40:46.188228 10164 server_base.cc:1034] running on GCE node
W20260806 11:40:46.188184 10172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:40:46.188172 10170 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:40:46.188196 10169 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:40:46.189085 10164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:40:46.190351 10164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:40:46.191553 10164 hybrid_clock.cc:648] HybridClock initialized: now 1786016446191508 us; error 60 us; skew 500 ppm
I20260806 11:40:46.193563 10164 webserver.cc:460] Webserver started at http://127.21.14.254:40551/ using document root <none> and password file <none>
I20260806 11:40:46.194211 10164 fs_manager.cc:359] Metadata directory not provided
I20260806 11:40:46.194310 10164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:40:46.194563 10164 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:40:46.196848 10164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786016169045816-21563-0/minicluster-data/master-0/data/instance:
uuid: "15f5876097144c2cb438b0338993df14"
format_stamp: "Formatted at 2026-08-06 11:40:46 on dist-test-slave-7c35"
server_key: "66ee9d03ad16de74c919de3bf59da10e"
server_key_iv: "55da82e1f89d2a2e0e34d75163141f57"
server_key_version: "encryptionkey@0"
I20260806 11:40:46.197455 10164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786016169045816-21563-0/minicluster-data/master-0/wal/instance:
uuid: "15f5876097144c2cb438b0338993df14"
format_stamp: "Formatted at 2026-08-06 11:40:46 on dist-test-slave-7c35"
server_key: "66ee9d03ad16de74c919de3bf59da10e"
server_key_iv: "55da82e1f89d2a2e0e34d75163141f57"
server_key_version: "encryptionkey@0"
I20260806 11:40:46.201396 10164 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260806 11:40:46.204161 10179 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:46.205734 10164 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 11:40:46.205919 10164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786016169045816-21563-0/minicluster-data/master-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786016169045816-21563-0/minicluster-data/master-0/wal
uuid: "15f5876097144c2cb438b0338993df14"
format_stamp: "Formatted at 2026-08-06 11:40:46 on dist-test-slave-7c35"
server_key: "66ee9d03ad16de74c919de3bf59da10e"
server_key_iv: "55da82e1f89d2a2e0e34d75163141f57"
server_key_version: "encryptionkey@0"
I20260806 11:40:46.206058 10164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786016169045816-21563-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786016169045816-21563-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786016169045816-21563-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:40:46.242937 10164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:40:46.243790 10164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:40:46.244035 10164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:40:46.252058 10164 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.254:36091
I20260806 11:40:46.252075 10234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.254:36091 every 8 connection(s)
I20260806 11:40:46.253123 10164 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
I20260806 11:40:46.256840 10235 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:40:46.263492 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 10164
I20260806 11:40:46.263667 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786016169045816-21563-0/minicluster-data/master-0/wal/instance
I20260806 11:40:46.263784 10235 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15f5876097144c2cb438b0338993df14: Bootstrap starting.
I20260806 11:40:46.264022 21563 external_mini_cluster.cc:1428] Setting key 4cc4b729873cf45ee333f411dfb78b24
I20260806 11:40:46.266959 10235 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 15f5876097144c2cb438b0338993df14: Neither blocks nor log segments found. Creating new log.
I20260806 11:40:46.267856 10235 log.cc:824] T 00000000000000000000000000000000 P 15f5876097144c2cb438b0338993df14: Log is configured to *not* fsync() on all Append() calls
I20260806 11:40:46.269976 10235 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15f5876097144c2cb438b0338993df14: No bootstrap required, opened a new log
I20260806 11:40:46.273952 10238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:46.265603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52554 (local address 127.21.14.254:36091)
0806 11:40:46.266359 (+ 756us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:46.266376 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:46.266415 (+ 39us) server_negotiation.cc:407] Connection header received
0806 11:40:46.267260 (+ 845us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:46.267283 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:46.267721 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:46.268077 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:40:46.268531 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:46.269519 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:46.270509 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:46.271056 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:46.271233 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:46.272157 (+ 924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:40:46.272162 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:40:46.272177 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 11:40:46.272588 (+ 411us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:40:46.272646 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 11:40:46.272667 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:40:46.272780 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:46.272898 (+ 118us) server_negotiation.cc:299] Negotiation successful
0806 11:40:46.273178 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":157,"threads_started":1}
I20260806 11:40:46.274226 10235 raft_consensus.cc:348] T 00000000000000000000000000000000 P 15f5876097144c2cb438b0338993df14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15f5876097144c2cb438b0338993df14" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 36091 } }
I20260806 11:40:46.274498 10235 raft_consensus.cc:374] T 00000000000000000000000000000000 P 15f5876097144c2cb438b0338993df14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:40:46.274582 10235 raft_consensus.cc:729] T 00000000000000000000000000000000 P 15f5876097144c2cb438b0338993df14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15f5876097144c2cb438b0338993df14, State: Initialized, Role: FOLLOWER
I20260806 11:40:46.275163 10235 consensus_queue.cc:260] T 00000000000000000000000000000000 P 15f5876097144c2cb438b0338993df14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15f5876097144c2cb438b0338993df14" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 36091 } }
I20260806 11:40:46.275338 10235 raft_consensus.cc:388] T 00000000000000000000000000000000 P 15f5876097144c2cb438b0338993df14 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 11:40:46.275399 10235 raft_consensus.cc:482] T 00000000000000000000000000000000 P 15f5876097144c2cb438b0338993df14 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 11:40:46.275499 10235 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 15f5876097144c2cb438b0338993df14 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:40:46.276780 10235 raft_consensus.cc:504] T 00000000000000000000000000000000 P 15f5876097144c2cb438b0338993df14 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15f5876097144c2cb438b0338993df14" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 36091 } }
I20260806 11:40:46.277101 10235 leader_election.cc:302] T 00000000000000000000000000000000 P 15f5876097144c2cb438b0338993df14 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 15f5876097144c2cb438b0338993df14; no voters:
I20260806 11:40:46.277398 10235 leader_election.cc:288] T 00000000000000000000000000000000 P 15f5876097144c2cb438b0338993df14 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 11:40:46.277557 10240 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 15f5876097144c2cb438b0338993df14 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:40:46.277805 10240 raft_consensus.cc:686] T 00000000000000000000000000000000 P 15f5876097144c2cb438b0338993df14 [term 1 LEADER]: Becoming Leader. State: Replica: 15f5876097144c2cb438b0338993df14, State: Running, Role: LEADER
I20260806 11:40:46.278139 10240 consensus_queue.cc:237] T 00000000000000000000000000000000 P 15f5876097144c2cb438b0338993df14 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15f5876097144c2cb438b0338993df14" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 36091 } }
I20260806 11:40:46.278744 10235 sys_catalog.cc:564] T 00000000000000000000000000000000 P 15f5876097144c2cb438b0338993df14 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 11:40:46.279968 10241 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15f5876097144c2cb438b0338993df14 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "15f5876097144c2cb438b0338993df14" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15f5876097144c2cb438b0338993df14" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 36091 } } }
I20260806 11:40:46.280107 10241 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15f5876097144c2cb438b0338993df14 [sys.catalog]: This master's current role is: LEADER
I20260806 11:40:46.280395 10242 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15f5876097144c2cb438b0338993df14 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 15f5876097144c2cb438b0338993df14. Latest consensus state: current_term: 1 leader_uuid: "15f5876097144c2cb438b0338993df14" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15f5876097144c2cb438b0338993df14" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 36091 } } }
I20260806 11:40:46.280484 10249 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 11:40:46.280501 10242 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15f5876097144c2cb438b0338993df14 [sys.catalog]: This master's current role is: LEADER
I20260806 11:40:46.283923 10249 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 11:40:46.290277 10249 catalog_manager.cc:1347] Generated new cluster ID: b9dd3c6c9c9546f8858fdcb4b5e08711
I20260806 11:40:46.290367 10249 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 11:40:46.306736 10249 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 11:40:46.308007 10249 catalog_manager.cc:1504] Loading token signing keys...
I20260806 11:40:46.316121 10249 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 15f5876097144c2cb438b0338993df14: Generated new TSK 0
I20260806 11:40:46.317113 10249 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 11:40:46.328325 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786016169045816-21563-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786016169045816-21563-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786016169045816-21563-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.193:0
--local_ip_for_outbound_sockets=127.21.14.193
--webserver_interface=127.21.14.193
--webserver_port=0
--tserver_master_addrs=127.21.14.254:36091
--builtin_ntp_servers=127.21.14.212:33153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 11:40:46.450191 10260 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:40:46.450484 10260 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:40:46.450525 10260 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:40:46.454716 10260 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:40:46.454790 10260 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:40:46.454869 10260 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.193
I20260806 11:40:46.459913 10260 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:33153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786016169045816-21563-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786016169045816-21563-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.14.193
--webserver_port=0
--tserver_master_addrs=127.21.14.254:36091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10260
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.193
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786016169045816-21563-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:40:46.461097 10260 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:40:46.462590 10260 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:40:46.472966 10265 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:40:46.472966 10268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:40:46.472998 10266 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:40:46.473407 10260 server_base.cc:1034] running on GCE node
I20260806 11:40:46.473855 10260 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:40:46.474556 10260 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:40:46.475768 10260 hybrid_clock.cc:648] HybridClock initialized: now 1786016446475740 us; error 34 us; skew 500 ppm
I20260806 11:40:46.477901 10260 webserver.cc:460] Webserver started at http://127.21.14.193:38741/ using document root <none> and password file <none>
I20260806 11:40:46.478545 10260 fs_manager.cc:359] Metadata directory not provided
I20260806 11:40:46.478611 10260 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:40:46.478837 10260 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:40:46.481357 10260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786016169045816-21563-0/minicluster-data/ts-0/data/instance:
uuid: "482f411b88f248d69a2dbd5f6a486201"
format_stamp: "Formatted at 2026-08-06 11:40:46 on dist-test-slave-7c35"
server_key: "c1a36f15c2e1145b081642ef434df59c"
server_key_iv: "9dcd83a83b8e0f6855bd985fd9025ee3"
server_key_version: "encryptionkey@0"
I20260806 11:40:46.481992 10260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786016169045816-21563-0/minicluster-data/ts-0/wal/instance:
uuid: "482f411b88f248d69a2dbd5f6a486201"
format_stamp: "Formatted at 2026-08-06 11:40:46 on dist-test-slave-7c35"
server_key: "c1a36f15c2e1145b081642ef434df59c"
server_key_iv: "9dcd83a83b8e0f6855bd985fd9025ee3"
server_key_version: "encryptionkey@0"
I20260806 11:40:46.486259 10260 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260806 11:40:46.489066 10274 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:46.490518 10260 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 11:40:46.490633 10260 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786016169045816-21563-0/minicluster-data/ts-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786016169045816-21563-0/minicluster-data/ts-0/wal
uuid: "482f411b88f248d69a2dbd5f6a486201"
format_stamp: "Formatted at 2026-08-06 11:40:46 on dist-test-slave-7c35"
server_key: "c1a36f15c2e1145b081642ef434df59c"
server_key_iv: "9dcd83a83b8e0f6855bd985fd9025ee3"
server_key_version: "encryptionkey@0"
I20260806 11:40:46.490782 10260 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786016169045816-21563-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786016169045816-21563-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786016169045816-21563-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:40:46.503168 10260 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:40:46.503932 10260 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:40:46.504138 10260 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:40:46.504781 10260 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:40:46.505788 10260 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:40:46.505836 10260 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:46.505901 10260 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:40:46.505934 10260 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:46.515532 10260 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.193:39271
I20260806 11:40:46.515573 10391 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.193:39271 every 8 connection(s)
I20260806 11:40:46.516654 10260 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786016169045816-21563-0/minicluster-data/ts-0/data/info.pb
I20260806 11:40:46.526970 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 10260
I20260806 11:40:46.527069 10238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:46.518723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.193:59057 (local address 127.21.14.254:36091)
0806 11:40:46.518902 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:46.518906 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:46.520059 (+ 1153us) server_negotiation.cc:407] Connection header received
0806 11:40:46.521531 (+ 1472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:46.521534 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:46.521588 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:46.521673 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:40:46.522687 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:46.523342 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:46.524635 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:46.524886 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:46.524988 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:46.526301 (+ 1313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:40:46.526304 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:40:46.526306 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 11:40:46.526323 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:40:46.526344 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 11:40:46.526353 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:40:46.526426 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:46.526755 (+ 329us) server_negotiation.cc:299] Negotiation successful
0806 11:40:46.526834 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 11:40:46.527122 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786016169045816-21563-0/minicluster-data/ts-0/wal/instance
I20260806 11:40:46.527420 21563 external_mini_cluster.cc:1428] Setting key eb89453fe8cb3e71223c68c56967dfb6
I20260806 11:40:46.528157 10394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:46.519236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:36091 (local address 127.21.14.193:59057)
0806 11:40:46.519922 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:46.519966 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 11:40:46.521288 (+ 1322us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:40:46.521854 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:40:46.521861 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:40:46.521954 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:40:46.522513 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:46.522529 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:46.523518 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:46.523522 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:46.524491 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:46.524503 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:46.525088 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:46.525091 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:46.525267 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:46.525977 (+ 710us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 11:40:46.525979 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:40:46.526037 (+ 58us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 11:40:46.526115 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:40:46.526484 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:40:46.526494 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:40:46.526500 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 11:40:46.526621 (+ 121us) client_negotiation.cc:240] Negotiation successful
0806 11:40:46.527006 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":122,"threads_started":1}
I20260806 11:40:46.529457 10392 heartbeater.cc:347] Connected to a master server at 127.21.14.254:36091
I20260806 11:40:46.529678 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786016169045816-21563-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786016169045816-21563-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786016169045816-21563-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.194:0
--local_ip_for_outbound_sockets=127.21.14.194
--webserver_interface=127.21.14.194
--webserver_port=0
--tserver_master_addrs=127.21.14.254:36091
--builtin_ntp_servers=127.21.14.212:33153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 11:40:46.529734 10392 heartbeater.cc:464] Registering TS with master...
I20260806 11:40:46.530478 10392 heartbeater.cc:511] Master 127.21.14.254:36091 requested a full tablet report, sending...
I20260806 11:40:46.533383 10198 ts_manager.cc:194] Registered new tserver with Master: 482f411b88f248d69a2dbd5f6a486201 (127.21.14.193:39271)
I20260806 11:40:46.535377 10198 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.14.193:59057
W20260806 11:40:46.650298 10395 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:40:46.650622 10395 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:40:46.650665 10395 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:40:46.654781 10395 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:40:46.654875 10395 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:40:46.655001 10395 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.194
I20260806 11:40:46.660662 10395 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:33153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786016169045816-21563-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786016169045816-21563-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.14.194
--webserver_port=0
--tserver_master_addrs=127.21.14.254:36091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10395
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.194
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786016169045816-21563-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:40:46.661816 10395 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:40:46.663298 10395 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:40:46.673838 10402 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:40:46.674021 10395 server_base.cc:1034] running on GCE node
W20260806 11:40:46.673838 10405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:40:46.673843 10403 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:40:46.674659 10395 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:40:46.675458 10395 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:40:46.676641 10395 hybrid_clock.cc:648] HybridClock initialized: now 1786016446676597 us; error 56 us; skew 500 ppm
I20260806 11:40:46.679023 10395 webserver.cc:460] Webserver started at http://127.21.14.194:41997/ using document root <none> and password file <none>
I20260806 11:40:46.679718 10395 fs_manager.cc:359] Metadata directory not provided
I20260806 11:40:46.679869 10395 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:40:46.680123 10395 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:40:46.682456 10395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786016169045816-21563-0/minicluster-data/ts-1/data/instance:
uuid: "23fa000cf3544e019f01e58fd2e28604"
format_stamp: "Formatted at 2026-08-06 11:40:46 on dist-test-slave-7c35"
server_key: "e93fc64be8696196a620510392f794ad"
server_key_iv: "637a4cd152ad1b39af41959026d6e1e5"
server_key_version: "encryptionkey@0"
I20260806 11:40:46.683053 10395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786016169045816-21563-0/minicluster-data/ts-1/wal/instance:
uuid: "23fa000cf3544e019f01e58fd2e28604"
format_stamp: "Formatted at 2026-08-06 11:40:46 on dist-test-slave-7c35"
server_key: "e93fc64be8696196a620510392f794ad"
server_key_iv: "637a4cd152ad1b39af41959026d6e1e5"
server_key_version: "encryptionkey@0"
I20260806 11:40:46.687121 10395 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 11:40:46.689743 10411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:46.691211 10395 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 11:40:46.691357 10395 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786016169045816-21563-0/minicluster-data/ts-1/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786016169045816-21563-0/minicluster-data/ts-1/wal
uuid: "23fa000cf3544e019f01e58fd2e28604"
format_stamp: "Formatted at 2026-08-06 11:40:46 on dist-test-slave-7c35"
server_key: "e93fc64be8696196a620510392f794ad"
server_key_iv: "637a4cd152ad1b39af41959026d6e1e5"
server_key_version: "encryptionkey@0"
I20260806 11:40:46.691488 10395 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786016169045816-21563-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786016169045816-21563-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786016169045816-21563-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:40:46.716862 10395 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:40:46.717737 10395 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:40:46.717993 10395 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:40:46.718611 10395 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:40:46.719619 10395 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:40:46.719666 10395 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:46.719708 10395 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:40:46.719730 10395 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:46.728802 10395 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.194:36355
I20260806 11:40:46.728843 10529 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.194:36355 every 8 connection(s)
I20260806 11:40:46.729772 10395 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786016169045816-21563-0/minicluster-data/ts-1/data/info.pb
I20260806 11:40:46.737129 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 10395
I20260806 11:40:46.737314 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786016169045816-21563-0/minicluster-data/ts-1/wal/instance
I20260806 11:40:46.737653 21563 external_mini_cluster.cc:1428] Setting key c315ec61c2434bbc8c0a7b29b8ddbe87
I20260806 11:40:46.740226 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786016169045816-21563-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786016169045816-21563-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786016169045816-21563-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.14.195:0
--local_ip_for_outbound_sockets=127.21.14.195
--webserver_interface=127.21.14.195
--webserver_port=0
--tserver_master_addrs=127.21.14.254:36091
--builtin_ntp_servers=127.21.14.212:33153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 11:40:46.740361 10238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:46.731933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:40987 (local address 127.21.14.254:36091)
0806 11:40:46.732110 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:46.732116 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:46.733171 (+ 1055us) server_negotiation.cc:407] Connection header received
0806 11:40:46.734597 (+ 1426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:46.734600 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:46.734651 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:46.734779 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:40:46.735867 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:46.736420 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:46.737851 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:46.738173 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:46.738268 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:46.739752 (+ 1484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:40:46.739757 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:40:46.739760 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 11:40:46.739774 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:40:46.739800 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 11:40:46.739811 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:40:46.739893 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:46.740107 (+ 214us) server_negotiation.cc:299] Negotiation successful
0806 11:40:46.740181 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 11:40:46.741348 10532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:46.732329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:36091 (local address 127.21.14.194:40987)
0806 11:40:46.733008 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:46.733050 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 11:40:46.734385 (+ 1335us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:40:46.735001 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:40:46.735008 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:40:46.735100 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:40:46.735659 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:46.735674 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:46.736575 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:46.736578 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:46.737687 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:46.737699 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:46.738391 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:46.738398 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:46.738600 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:46.739423 (+ 823us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 11:40:46.739426 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:40:46.739495 (+ 69us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 11:40:46.739576 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:40:46.739928 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:40:46.739939 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:40:46.739944 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 11:40:46.740068 (+ 124us) client_negotiation.cc:240] Negotiation successful
0806 11:40:46.740390 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":133,"threads_started":1}
I20260806 11:40:46.742795 10530 heartbeater.cc:347] Connected to a master server at 127.21.14.254:36091
I20260806 11:40:46.743101 10530 heartbeater.cc:464] Registering TS with master...
I20260806 11:40:46.743799 10530 heartbeater.cc:511] Master 127.21.14.254:36091 requested a full tablet report, sending...
I20260806 11:40:46.745056 10198 ts_manager.cc:194] Registered new tserver with Master: 23fa000cf3544e019f01e58fd2e28604 (127.21.14.194:36355)
I20260806 11:40:46.745682 10198 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.14.194:40987
W20260806 11:40:46.863595 10533 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:40:46.863903 10533 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:40:46.863945 10533 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:40:46.868335 10533 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:40:46.868403 10533 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:40:46.868463 10533 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.14.195
I20260806 11:40:46.873575 10533 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:33153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786016169045816-21563-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786016169045816-21563-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.14.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.14.195
--webserver_port=0
--tserver_master_addrs=127.21.14.254:36091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10533
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.14.195
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786016169045816-21563-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:40:46.874871 10533 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:40:46.876370 10533 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:40:46.887444 10539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:40:46.887617 10542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:40:46.887444 10538 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:40:46.887848 10533 server_base.cc:1034] running on GCE node
I20260806 11:40:46.888309 10533 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:40:46.889034 10533 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:40:46.890239 10533 hybrid_clock.cc:648] HybridClock initialized: now 1786016446890212 us; error 38 us; skew 500 ppm
I20260806 11:40:46.892413 10533 webserver.cc:460] Webserver started at http://127.21.14.195:35985/ using document root <none> and password file <none>
I20260806 11:40:46.893077 10533 fs_manager.cc:359] Metadata directory not provided
I20260806 11:40:46.893149 10533 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:40:46.893390 10533 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:40:46.895915 10533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786016169045816-21563-0/minicluster-data/ts-2/data/instance:
uuid: "571412cc94054332be01c037c92139a3"
format_stamp: "Formatted at 2026-08-06 11:40:46 on dist-test-slave-7c35"
server_key: "d86a4ea2c484992c464932f46b3ad8e9"
server_key_iv: "4d3f6cd344d364b0260680501abfcc16"
server_key_version: "encryptionkey@0"
I20260806 11:40:46.896541 10533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786016169045816-21563-0/minicluster-data/ts-2/wal/instance:
uuid: "571412cc94054332be01c037c92139a3"
format_stamp: "Formatted at 2026-08-06 11:40:46 on dist-test-slave-7c35"
server_key: "d86a4ea2c484992c464932f46b3ad8e9"
server_key_iv: "4d3f6cd344d364b0260680501abfcc16"
server_key_version: "encryptionkey@0"
I20260806 11:40:46.900580 10533 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 11:40:46.903271 10549 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:46.904582 10533 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 11:40:46.904733 10533 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786016169045816-21563-0/minicluster-data/ts-2/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786016169045816-21563-0/minicluster-data/ts-2/wal
uuid: "571412cc94054332be01c037c92139a3"
format_stamp: "Formatted at 2026-08-06 11:40:46 on dist-test-slave-7c35"
server_key: "d86a4ea2c484992c464932f46b3ad8e9"
server_key_iv: "4d3f6cd344d364b0260680501abfcc16"
server_key_version: "encryptionkey@0"
I20260806 11:40:46.904862 10533 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786016169045816-21563-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786016169045816-21563-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786016169045816-21563-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:40:46.933244 10533 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:40:46.934067 10533 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:40:46.934299 10533 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:40:46.935043 10533 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 11:40:46.936102 10533 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 11:40:46.936182 10533 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:46.936262 10533 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 11:40:46.936307 10533 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:46.945536 10533 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.195:34939
I20260806 11:40:46.945567 10668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.195:34939 every 8 connection(s)
I20260806 11:40:46.946521 10533 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786016169045816-21563-0/minicluster-data/ts-2/data/info.pb
I20260806 11:40:46.949277 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 10533
I20260806 11:40:46.949427 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786016169045816-21563-0/minicluster-data/ts-2/wal/instance
I20260806 11:40:46.949748 21563 external_mini_cluster.cc:1428] Setting key f2406488eeaeb3066c6318de4110f2c3
I20260806 11:40:46.957118 10238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:46.948536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.195:33367 (local address 127.21.14.254:36091)
0806 11:40:46.948686 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:46.948691 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:46.949858 (+ 1167us) server_negotiation.cc:407] Connection header received
0806 11:40:46.951803 (+ 1945us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:46.951806 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:46.951858 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:46.951942 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:40:46.953009 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:46.953571 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:46.955009 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:46.955239 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:46.955333 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:46.956568 (+ 1235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:40:46.956571 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:40:46.956573 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 11:40:46.956590 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:40:46.956610 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 11:40:46.956619 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:40:46.956686 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:46.956921 (+ 235us) server_negotiation.cc:299] Negotiation successful
0806 11:40:46.956986 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 11:40:46.958026 10671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:46.948922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:36091 (local address 127.21.14.195:33367)
0806 11:40:46.949693 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:46.949749 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 11:40:46.951555 (+ 1806us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:40:46.952171 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:40:46.952181 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:40:46.952299 (+ 118us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:40:46.952849 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:46.952863 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:46.953735 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:46.953738 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:46.954787 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:46.954799 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:46.955426 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:46.955431 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:46.955592 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:46.956272 (+ 680us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 11:40:46.956274 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:40:46.956321 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 11:40:46.956394 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:40:46.956732 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:40:46.956741 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:40:46.956747 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 11:40:46.956871 (+ 124us) client_negotiation.cc:240] Negotiation successful
0806 11:40:46.957143 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":164,"threads_started":1}
I20260806 11:40:46.959394 10669 heartbeater.cc:347] Connected to a master server at 127.21.14.254:36091
I20260806 11:40:46.959683 10669 heartbeater.cc:464] Registering TS with master...
I20260806 11:40:46.960287 10669 heartbeater.cc:511] Master 127.21.14.254:36091 requested a full tablet report, sending...
I20260806 11:40:46.961448 10198 ts_manager.cc:194] Registered new tserver with Master: 571412cc94054332be01c037c92139a3 (127.21.14.195:34939)
I20260806 11:40:46.962071 10198 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.14.195:33367
I20260806 11:40:46.964425 21563 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 11:40:46.970435 10238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:46.966440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52566 (local address 127.21.14.254:36091)
0806 11:40:46.966600 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:46.966607 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:46.966784 (+ 177us) server_negotiation.cc:407] Connection header received
0806 11:40:46.966956 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:46.966960 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:46.967020 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:46.967110 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:40:46.967408 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:46.967938 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:46.968805 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:46.969037 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:46.969149 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:46.969959 (+ 810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:40:46.969963 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:40:46.969966 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 11:40:46.969986 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:40:46.970008 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 11:40:46.970016 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:40:46.970108 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:46.970226 (+ 118us) server_negotiation.cc:299] Negotiation successful
0806 11:40:46.970287 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 11:40:46.973426 10198 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52566:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 11:40:46.976675 10198 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 11:40:46.993168 10680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:46.986892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.193:39271 (local address 127.0.0.1:42182)
0806 11:40:46.987916 (+ 1024us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:46.987929 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 11:40:46.988009 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:40:46.988573 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:40:46.988576 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:40:46.988592 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:40:46.988696 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:46.988701 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:46.989891 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:46.989895 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:46.991406 (+ 1511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:46.991418 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:46.992526 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:46.992533 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:46.992646 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:46.992687 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 11:40:46.992809 (+ 122us) client_negotiation.cc:240] Negotiation successful
0806 11:40:46.992954 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":967,"mutex_wait_us":127,"thread_start_us":94,"threads_started":1}
I20260806 11:40:46.993593 10682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:46.986274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38486 (local address 127.21.14.194:36355)
0806 11:40:46.986673 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:46.986687 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:46.987705 (+ 1018us) server_negotiation.cc:407] Connection header received
0806 11:40:46.987848 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:46.987855 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:46.988080 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:46.988238 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:40:46.988690 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:46.989857 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:46.991847 (+ 1990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:46.992453 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:46.992571 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:46.992776 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:46.993165 (+ 389us) server_negotiation.cc:299] Negotiation successful
0806 11:40:46.993415 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":181,"threads_started":1}
I20260806 11:40:46.993629 10684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:46.986122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.195:34939 (local address 127.0.0.1:33810)
0806 11:40:46.987429 (+ 1307us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:46.987488 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 11:40:46.987654 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:40:46.988251 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:40:46.988257 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:40:46.988278 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:40:46.988419 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:46.988427 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:46.989980 (+ 1553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:46.989983 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:46.991674 (+ 1691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:46.991686 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:46.992803 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:46.992807 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:46.992919 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:46.993020 (+ 101us) client_negotiation.cc:769] Sending connection context
0806 11:40:46.993093 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 11:40:46.993166 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1140,"mutex_wait_us":216,"thread_start_us":122,"threads_started":1}
I20260806 11:40:46.994081 10683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:46.986834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33810 (local address 127.21.14.195:34939)
0806 11:40:46.987296 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:46.987307 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:46.987622 (+ 315us) server_negotiation.cc:407] Connection header received
0806 11:40:46.987841 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:46.987848 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:46.988080 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:46.988248 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:40:46.988561 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:46.989830 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:46.991819 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:46.992616 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:46.992719 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:46.992898 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:46.993256 (+ 358us) server_negotiation.cc:299] Negotiation successful
0806 11:40:46.993917 (+ 661us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":107,"threads_started":1}
I20260806 11:40:46.994220 10681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:46.986104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.194:36355 (local address 127.0.0.1:38486)
0806 11:40:46.987600 (+ 1496us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:46.987616 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 11:40:46.987729 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:40:46.988444 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:40:46.988447 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:40:46.988464 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:40:46.988561 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:46.988567 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:46.989980 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:46.989983 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:46.991674 (+ 1691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:46.991686 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:46.992841 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:46.992844 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:46.992966 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:46.993023 (+ 57us) client_negotiation.cc:769] Sending connection context
0806 11:40:46.993940 (+ 917us) client_negotiation.cc:240] Negotiation successful
0806 11:40:46.994062 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":780,"mutex_wait_us":199,"thread_start_us":84,"threads_started":1}
I20260806 11:40:46.994297 10685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:46.987279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42182 (local address 127.21.14.193:39271)
0806 11:40:46.988014 (+ 735us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:46.988025 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:46.988050 (+ 25us) server_negotiation.cc:407] Connection header received
0806 11:40:46.988126 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:46.988134 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:46.988295 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:46.988433 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:40:46.988822 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:46.989737 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:46.991543 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:46.992365 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:46.992491 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:46.992721 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:46.993010 (+ 289us) server_negotiation.cc:299] Negotiation successful
0806 11:40:46.994128 (+ 1118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":624,"thread_start_us":107,"threads_started":1}
I20260806 11:40:46.997257 10323 tablet_service.cc:1467] Processing CreateTablet for tablet 2d3015cd13a54b819551ad6bb18925ff (DEFAULT_TABLE table=test-table [id=07a22dc59a6d46ba8eaab173f66b287e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:40:46.997254 10598 tablet_service.cc:1467] Processing CreateTablet for tablet 2d3015cd13a54b819551ad6bb18925ff (DEFAULT_TABLE table=test-table [id=07a22dc59a6d46ba8eaab173f66b287e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:40:46.997259 10460 tablet_service.cc:1467] Processing CreateTablet for tablet 2d3015cd13a54b819551ad6bb18925ff (DEFAULT_TABLE table=test-table [id=07a22dc59a6d46ba8eaab173f66b287e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 11:40:46.998476 10460 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d3015cd13a54b819551ad6bb18925ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:40:46.998497 10598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d3015cd13a54b819551ad6bb18925ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:40:46.998497 10323 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d3015cd13a54b819551ad6bb18925ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:40:47.005518 10686 tablet_bootstrap.cc:492] T 2d3015cd13a54b819551ad6bb18925ff P 23fa000cf3544e019f01e58fd2e28604: Bootstrap starting.
I20260806 11:40:47.006691 10688 tablet_bootstrap.cc:492] T 2d3015cd13a54b819551ad6bb18925ff P 571412cc94054332be01c037c92139a3: Bootstrap starting.
I20260806 11:40:47.007929 10686 tablet_bootstrap.cc:654] T 2d3015cd13a54b819551ad6bb18925ff P 23fa000cf3544e019f01e58fd2e28604: Neither blocks nor log segments found. Creating new log.
I20260806 11:40:47.008718 10688 tablet_bootstrap.cc:654] T 2d3015cd13a54b819551ad6bb18925ff P 571412cc94054332be01c037c92139a3: Neither blocks nor log segments found. Creating new log.
I20260806 11:40:47.008956 10686 log.cc:824] T 2d3015cd13a54b819551ad6bb18925ff P 23fa000cf3544e019f01e58fd2e28604: Log is configured to *not* fsync() on all Append() calls
I20260806 11:40:47.009569 10688 log.cc:824] T 2d3015cd13a54b819551ad6bb18925ff P 571412cc94054332be01c037c92139a3: Log is configured to *not* fsync() on all Append() calls
I20260806 11:40:47.009824 10687 tablet_bootstrap.cc:492] T 2d3015cd13a54b819551ad6bb18925ff P 482f411b88f248d69a2dbd5f6a486201: Bootstrap starting.
I20260806 11:40:47.011588 10688 tablet_bootstrap.cc:492] T 2d3015cd13a54b819551ad6bb18925ff P 571412cc94054332be01c037c92139a3: No bootstrap required, opened a new log
I20260806 11:40:47.011632 10686 tablet_bootstrap.cc:492] T 2d3015cd13a54b819551ad6bb18925ff P 23fa000cf3544e019f01e58fd2e28604: No bootstrap required, opened a new log
I20260806 11:40:47.011821 10688 ts_tablet_manager.cc:1397] T 2d3015cd13a54b819551ad6bb18925ff P 571412cc94054332be01c037c92139a3: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 11:40:47.011819 10686 ts_tablet_manager.cc:1397] T 2d3015cd13a54b819551ad6bb18925ff P 23fa000cf3544e019f01e58fd2e28604: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 11:40:47.012320 10687 tablet_bootstrap.cc:654] T 2d3015cd13a54b819551ad6bb18925ff P 482f411b88f248d69a2dbd5f6a486201: Neither blocks nor log segments found. Creating new log.
I20260806 11:40:47.013230 10687 log.cc:824] T 2d3015cd13a54b819551ad6bb18925ff P 482f411b88f248d69a2dbd5f6a486201: Log is configured to *not* fsync() on all Append() calls
I20260806 11:40:47.015949 10687 tablet_bootstrap.cc:492] T 2d3015cd13a54b819551ad6bb18925ff P 482f411b88f248d69a2dbd5f6a486201: No bootstrap required, opened a new log
I20260806 11:40:47.016179 10687 ts_tablet_manager.cc:1397] T 2d3015cd13a54b819551ad6bb18925ff P 482f411b88f248d69a2dbd5f6a486201: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 11:40:47.016247 10688 raft_consensus.cc:348] T 2d3015cd13a54b819551ad6bb18925ff P 571412cc94054332be01c037c92139a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23fa000cf3544e019f01e58fd2e28604" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 36355 } } peers { permanent_uuid: "571412cc94054332be01c037c92139a3" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 34939 } } peers { permanent_uuid: "482f411b88f248d69a2dbd5f6a486201" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 39271 } }
I20260806 11:40:47.016268 10686 raft_consensus.cc:348] T 2d3015cd13a54b819551ad6bb18925ff P 23fa000cf3544e019f01e58fd2e28604 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23fa000cf3544e019f01e58fd2e28604" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 36355 } } peers { permanent_uuid: "571412cc94054332be01c037c92139a3" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 34939 } } peers { permanent_uuid: "482f411b88f248d69a2dbd5f6a486201" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 39271 } }
I20260806 11:40:47.016582 10686 raft_consensus.cc:374] T 2d3015cd13a54b819551ad6bb18925ff P 23fa000cf3544e019f01e58fd2e28604 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:40:47.016583 10688 raft_consensus.cc:374] T 2d3015cd13a54b819551ad6bb18925ff P 571412cc94054332be01c037c92139a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:40:47.016649 10688 raft_consensus.cc:729] T 2d3015cd13a54b819551ad6bb18925ff P 571412cc94054332be01c037c92139a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 571412cc94054332be01c037c92139a3, State: Initialized, Role: FOLLOWER
I20260806 11:40:47.016650 10686 raft_consensus.cc:729] T 2d3015cd13a54b819551ad6bb18925ff P 23fa000cf3544e019f01e58fd2e28604 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23fa000cf3544e019f01e58fd2e28604, State: Initialized, Role: FOLLOWER
I20260806 11:40:47.017148 10688 consensus_queue.cc:260] T 2d3015cd13a54b819551ad6bb18925ff P 571412cc94054332be01c037c92139a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23fa000cf3544e019f01e58fd2e28604" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 36355 } } peers { permanent_uuid: "571412cc94054332be01c037c92139a3" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 34939 } } peers { permanent_uuid: "482f411b88f248d69a2dbd5f6a486201" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 39271 } }
I20260806 11:40:47.017153 10686 consensus_queue.cc:260] T 2d3015cd13a54b819551ad6bb18925ff P 23fa000cf3544e019f01e58fd2e28604 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23fa000cf3544e019f01e58fd2e28604" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 36355 } } peers { permanent_uuid: "571412cc94054332be01c037c92139a3" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 34939 } } peers { permanent_uuid: "482f411b88f248d69a2dbd5f6a486201" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 39271 } }
I20260806 11:40:47.017984 10530 heartbeater.cc:503] Master 127.21.14.254:36091 was elected leader, sending a full tablet report...
I20260806 11:40:47.018239 10669 heartbeater.cc:503] Master 127.21.14.254:36091 was elected leader, sending a full tablet report...
I20260806 11:40:47.018318 10688 ts_tablet_manager.cc:1428] T 2d3015cd13a54b819551ad6bb18925ff P 571412cc94054332be01c037c92139a3: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 11:40:47.018301 10686 ts_tablet_manager.cc:1428] T 2d3015cd13a54b819551ad6bb18925ff P 23fa000cf3544e019f01e58fd2e28604: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 11:40:47.022969 10687 raft_consensus.cc:348] T 2d3015cd13a54b819551ad6bb18925ff P 482f411b88f248d69a2dbd5f6a486201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23fa000cf3544e019f01e58fd2e28604" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 36355 } } peers { permanent_uuid: "571412cc94054332be01c037c92139a3" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 34939 } } peers { permanent_uuid: "482f411b88f248d69a2dbd5f6a486201" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 39271 } }
I20260806 11:40:47.023257 10687 raft_consensus.cc:374] T 2d3015cd13a54b819551ad6bb18925ff P 482f411b88f248d69a2dbd5f6a486201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:40:47.023313 10687 raft_consensus.cc:729] T 2d3015cd13a54b819551ad6bb18925ff P 482f411b88f248d69a2dbd5f6a486201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482f411b88f248d69a2dbd5f6a486201, State: Initialized, Role: FOLLOWER
I20260806 11:40:47.023856 10687 consensus_queue.cc:260] T 2d3015cd13a54b819551ad6bb18925ff P 482f411b88f248d69a2dbd5f6a486201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23fa000cf3544e019f01e58fd2e28604" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 36355 } } peers { permanent_uuid: "571412cc94054332be01c037c92139a3" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 34939 } } peers { permanent_uuid: "482f411b88f248d69a2dbd5f6a486201" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 39271 } }
I20260806 11:40:47.024710 10392 heartbeater.cc:503] Master 127.21.14.254:36091 was elected leader, sending a full tablet report...
I20260806 11:40:47.025105 10687 ts_tablet_manager.cc:1428] T 2d3015cd13a54b819551ad6bb18925ff P 482f411b88f248d69a2dbd5f6a486201: Time spent starting tablet: real 0.009s user 0.007s sys 0.000s
I20260806 11:40:47.031546 10692 raft_consensus.cc:482] T 2d3015cd13a54b819551ad6bb18925ff P 23fa000cf3544e019f01e58fd2e28604 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 11:40:47.031746 10692 raft_consensus.cc:504] T 2d3015cd13a54b819551ad6bb18925ff P 23fa000cf3544e019f01e58fd2e28604 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23fa000cf3544e019f01e58fd2e28604" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 36355 } } peers { permanent_uuid: "571412cc94054332be01c037c92139a3" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 34939 } } peers { permanent_uuid: "482f411b88f248d69a2dbd5f6a486201" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 39271 } }
I20260806 11:40:47.032835 10692 leader_election.cc:288] T 2d3015cd13a54b819551ad6bb18925ff P 23fa000cf3544e019f01e58fd2e28604 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 571412cc94054332be01c037c92139a3 (127.21.14.195:34939), 482f411b88f248d69a2dbd5f6a486201 (127.21.14.193:39271)
I20260806 11:40:47.037125 10683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:47.033283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:59289 (local address 127.21.14.195:34939)
0806 11:40:47.033417 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:47.033421 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:47.033438 (+ 17us) server_negotiation.cc:407] Connection header received
0806 11:40:47.033491 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:47.033493 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:47.033532 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:47.033643 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:40:47.033973 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:47.034529 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:47.036026 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:47.036610 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:47.036707 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:47.036748 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:47.036951 (+ 203us) server_negotiation.cc:299] Negotiation successful
0806 11:40:47.037009 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 11:40:47.037144 10532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:47.033138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.195:34939 (local address 127.21.14.194:59289)
0806 11:40:47.033278 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:47.033294 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 11:40:47.033401 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:40:47.033673 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:40:47.033676 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:40:47.033697 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:40:47.033828 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:47.033835 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:47.034672 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:47.034675 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:47.035882 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:47.035893 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:47.036706 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:47.036710 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:47.036817 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:47.036843 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 11:40:47.036925 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 11:40:47.037000 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260806 11:40:47.037722 10685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:47.033690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.14.194:56837 (local address 127.21.14.193:39271)
0806 11:40:47.033833 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:47.033836 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:47.033919 (+ 83us) server_negotiation.cc:407] Connection header received
0806 11:40:47.034047 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:47.034050 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:47.034093 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:47.034174 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 11:40:47.034474 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:47.035163 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:47.036462 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:47.037076 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:47.037204 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:47.037255 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:47.037518 (+ 263us) server_negotiation.cc:299] Negotiation successful
0806 11:40:47.037606 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 11:40:47.037736 10696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:47.033416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.193:39271 (local address 127.21.14.194:56837)
0806 11:40:47.033835 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:47.033848 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 11:40:47.033942 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:40:47.034198 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:40:47.034201 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:40:47.034217 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 11:40:47.034334 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:47.034339 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:47.035283 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:47.035290 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:47.036329 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:40:47.036342 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:40:47.037241 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:40:47.037245 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:40:47.037358 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:47.037391 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 11:40:47.037488 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 11:40:47.037571 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":146,"threads_started":1}
I20260806 11:40:47.037931 10618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d3015cd13a54b819551ad6bb18925ff" candidate_uuid: "23fa000cf3544e019f01e58fd2e28604" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "571412cc94054332be01c037c92139a3" is_pre_election: true
I20260806 11:40:47.038318 10618 raft_consensus.cc:2454] T 2d3015cd13a54b819551ad6bb18925ff P 571412cc94054332be01c037c92139a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23fa000cf3544e019f01e58fd2e28604 in term 0.
I20260806 11:40:47.038457 10345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d3015cd13a54b819551ad6bb18925ff" candidate_uuid: "23fa000cf3544e019f01e58fd2e28604" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482f411b88f248d69a2dbd5f6a486201" is_pre_election: true
I20260806 11:40:47.038756 10345 raft_consensus.cc:2454] T 2d3015cd13a54b819551ad6bb18925ff P 482f411b88f248d69a2dbd5f6a486201 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23fa000cf3544e019f01e58fd2e28604 in term 0.
I20260806 11:40:47.038887 10415 leader_election.cc:302] T 2d3015cd13a54b819551ad6bb18925ff P 23fa000cf3544e019f01e58fd2e28604 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23fa000cf3544e019f01e58fd2e28604, 571412cc94054332be01c037c92139a3; no voters:
I20260806 11:40:47.039235 10692 raft_consensus.cc:2781] T 2d3015cd13a54b819551ad6bb18925ff P 23fa000cf3544e019f01e58fd2e28604 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 11:40:47.039330 10692 raft_consensus.cc:482] T 2d3015cd13a54b819551ad6bb18925ff P 23fa000cf3544e019f01e58fd2e28604 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 11:40:47.039361 10692 raft_consensus.cc:3037] T 2d3015cd13a54b819551ad6bb18925ff P 23fa000cf3544e019f01e58fd2e28604 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:40:47.040445 10692 raft_consensus.cc:504] T 2d3015cd13a54b819551ad6bb18925ff P 23fa000cf3544e019f01e58fd2e28604 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23fa000cf3544e019f01e58fd2e28604" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 36355 } } peers { permanent_uuid: "571412cc94054332be01c037c92139a3" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 34939 } } peers { permanent_uuid: "482f411b88f248d69a2dbd5f6a486201" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 39271 } }
I20260806 11:40:47.040814 10692 leader_election.cc:288] T 2d3015cd13a54b819551ad6bb18925ff P 23fa000cf3544e019f01e58fd2e28604 [CANDIDATE]: Term 1 election: Requested vote from peers 571412cc94054332be01c037c92139a3 (127.21.14.195:34939), 482f411b88f248d69a2dbd5f6a486201 (127.21.14.193:39271)
I20260806 11:40:47.041337 10345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d3015cd13a54b819551ad6bb18925ff" candidate_uuid: "23fa000cf3544e019f01e58fd2e28604" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482f411b88f248d69a2dbd5f6a486201"
I20260806 11:40:47.041337 10618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d3015cd13a54b819551ad6bb18925ff" candidate_uuid: "23fa000cf3544e019f01e58fd2e28604" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "571412cc94054332be01c037c92139a3"
I20260806 11:40:47.041489 10345 raft_consensus.cc:3037] T 2d3015cd13a54b819551ad6bb18925ff P 482f411b88f248d69a2dbd5f6a486201 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:40:47.041522 10618 raft_consensus.cc:3037] T 2d3015cd13a54b819551ad6bb18925ff P 571412cc94054332be01c037c92139a3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:40:47.042948 10618 raft_consensus.cc:2454] T 2d3015cd13a54b819551ad6bb18925ff P 571412cc94054332be01c037c92139a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23fa000cf3544e019f01e58fd2e28604 in term 1.
I20260806 11:40:47.042948 10345 raft_consensus.cc:2454] T 2d3015cd13a54b819551ad6bb18925ff P 482f411b88f248d69a2dbd5f6a486201 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23fa000cf3544e019f01e58fd2e28604 in term 1.
I20260806 11:40:47.043298 10415 leader_election.cc:302] T 2d3015cd13a54b819551ad6bb18925ff P 23fa000cf3544e019f01e58fd2e28604 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23fa000cf3544e019f01e58fd2e28604, 571412cc94054332be01c037c92139a3; no voters:
I20260806 11:40:47.043480 10692 raft_consensus.cc:2781] T 2d3015cd13a54b819551ad6bb18925ff P 23fa000cf3544e019f01e58fd2e28604 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:40:47.043717 10692 raft_consensus.cc:686] T 2d3015cd13a54b819551ad6bb18925ff P 23fa000cf3544e019f01e58fd2e28604 [term 1 LEADER]: Becoming Leader. State: Replica: 23fa000cf3544e019f01e58fd2e28604, State: Running, Role: LEADER
I20260806 11:40:47.044116 10692 consensus_queue.cc:237] T 2d3015cd13a54b819551ad6bb18925ff P 23fa000cf3544e019f01e58fd2e28604 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23fa000cf3544e019f01e58fd2e28604" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 36355 } } peers { permanent_uuid: "571412cc94054332be01c037c92139a3" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 34939 } } peers { permanent_uuid: "482f411b88f248d69a2dbd5f6a486201" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 39271 } }
I20260806 11:40:47.047474 10198 catalog_manager.cc:5495] T 2d3015cd13a54b819551ad6bb18925ff P 23fa000cf3544e019f01e58fd2e28604 reported cstate change: term changed from 0 to 1, leader changed from <none> to 23fa000cf3544e019f01e58fd2e28604 (127.21.14.194). New cstate: current_term: 1 leader_uuid: "23fa000cf3544e019f01e58fd2e28604" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23fa000cf3544e019f01e58fd2e28604" member_type: VOTER last_known_addr { host: "127.21.14.194" port: 36355 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "571412cc94054332be01c037c92139a3" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 34939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "482f411b88f248d69a2dbd5f6a486201" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 39271 } health_report { overall_health: UNKNOWN } } }
I20260806 11:40:47.073115 10682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:47.069269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38502 (local address 127.21.14.194:36355)
0806 11:40:47.069405 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:47.069409 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:47.069493 (+ 84us) server_negotiation.cc:407] Connection header received
0806 11:40:47.069648 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:47.069652 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:47.069731 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:47.069846 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 11:40:47.070105 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:47.070725 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:47.071710 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:47.071927 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:47.072018 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:47.072319 (+ 301us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 11:40:47.072770 (+ 451us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 11:40:47.072868 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:47.072934 (+ 66us) server_negotiation.cc:299] Negotiation successful
0806 11:40:47.072987 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 11:40:47.079761 10345 raft_consensus.cc:1261] T 2d3015cd13a54b819551ad6bb18925ff P 482f411b88f248d69a2dbd5f6a486201 [term 1 FOLLOWER]: Refusing update from remote peer 23fa000cf3544e019f01e58fd2e28604: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 11:40:47.079749 10618 raft_consensus.cc:1261] T 2d3015cd13a54b819551ad6bb18925ff P 571412cc94054332be01c037c92139a3 [term 1 FOLLOWER]: Refusing update from remote peer 23fa000cf3544e019f01e58fd2e28604: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 11:40:47.080641 10692 consensus_queue.cc:1035] T 2d3015cd13a54b819551ad6bb18925ff P 23fa000cf3544e019f01e58fd2e28604 [LEADER]: Connected to new peer: Peer: permanent_uuid: "482f411b88f248d69a2dbd5f6a486201" member_type: VOTER last_known_addr { host: "127.21.14.193" port: 39271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 11:40:47.080880 10697 consensus_queue.cc:1035] T 2d3015cd13a54b819551ad6bb18925ff P 23fa000cf3544e019f01e58fd2e28604 [LEADER]: Connected to new peer: Peer: permanent_uuid: "571412cc94054332be01c037c92139a3" member_type: VOTER last_known_addr { host: "127.21.14.195" port: 34939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 11:40:47.090907 10701 mvcc.cc:205] Tried to move back new op lower bound from 7315523367229718528 to 7315523367095357440. Current Snapshot: MvccSnapshot[applied={T|T < 7315523367229718528}]
I20260806 11:40:47.091938 10702 mvcc.cc:205] Tried to move back new op lower bound from 7315523367229718528 to 7315523367095357440. Current Snapshot: MvccSnapshot[applied={T|T < 7315523367229718528}]
I20260806 11:40:47.090907 10700 mvcc.cc:205] Tried to move back new op lower bound from 7315523367229718528 to 7315523367095357440. Current Snapshot: MvccSnapshot[applied={T|T < 7315523367229718528}]
I20260806 11:40:47.100150 10196 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52566:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 11:40:47.100378 10196 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:52566:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 11:40:47.103426 10196 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 15f5876097144c2cb438b0338993df14: Sending DeleteTablet for 3 replicas of tablet 2d3015cd13a54b819551ad6bb18925ff
I20260806 11:40:47.104446 10460 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d3015cd13a54b819551ad6bb18925ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:40:47 UTC) from {username='slave'} at 127.0.0.1:38486
I20260806 11:40:47.104522 10598 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d3015cd13a54b819551ad6bb18925ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:40:47 UTC) from {username='slave'} at 127.0.0.1:33810
I20260806 11:40:47.105125 10710 tablet_replica.cc:330] T 2d3015cd13a54b819551ad6bb18925ff P 23fa000cf3544e019f01e58fd2e28604: stopping tablet replica
I20260806 11:40:47.105324 10323 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d3015cd13a54b819551ad6bb18925ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 11:40:47 UTC) from {username='slave'} at 127.0.0.1:42182
I20260806 11:40:47.105676 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 10260
I20260806 11:40:47.106487 10711 tablet_replica.cc:330] T 2d3015cd13a54b819551ad6bb18925ff P 571412cc94054332be01c037c92139a3: stopping tablet replica
I20260806 11:40:47.106897 10711 raft_consensus.cc:2229] T 2d3015cd13a54b819551ad6bb18925ff P 571412cc94054332be01c037c92139a3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 11:40:47.107298 10711 raft_consensus.cc:2258] T 2d3015cd13a54b819551ad6bb18925ff P 571412cc94054332be01c037c92139a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 11:40:47.105679 10710 raft_consensus.cc:2229] T 2d3015cd13a54b819551ad6bb18925ff P 23fa000cf3544e019f01e58fd2e28604 [term 1 LEADER]: Raft consensus shutting down.
I20260806 11:40:47.109638 10710 raft_consensus.cc:2258] T 2d3015cd13a54b819551ad6bb18925ff P 23fa000cf3544e019f01e58fd2e28604 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 11:40:47.110986 10711 ts_tablet_manager.cc:1905] T 2d3015cd13a54b819551ad6bb18925ff P 571412cc94054332be01c037c92139a3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 11:40:47.111498 10710 ts_tablet_manager.cc:1905] T 2d3015cd13a54b819551ad6bb18925ff P 23fa000cf3544e019f01e58fd2e28604: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 11:40:47.114372 10711 ts_tablet_manager.cc:1918] T 2d3015cd13a54b819551ad6bb18925ff P 571412cc94054332be01c037c92139a3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 11:40:47.114372 10710 ts_tablet_manager.cc:1918] T 2d3015cd13a54b819551ad6bb18925ff P 23fa000cf3544e019f01e58fd2e28604: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 11:40:47.114502 10710 log.cc:1196] T 2d3015cd13a54b819551ad6bb18925ff P 23fa000cf3544e019f01e58fd2e28604: Deleting WAL directory at /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786016169045816-21563-0/minicluster-data/ts-1/wal/wals/2d3015cd13a54b819551ad6bb18925ff
I20260806 11:40:47.114667 10711 log.cc:1196] T 2d3015cd13a54b819551ad6bb18925ff P 571412cc94054332be01c037c92139a3: Deleting WAL directory at /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786016169045816-21563-0/minicluster-data/ts-2/wal/wals/2d3015cd13a54b819551ad6bb18925ff
I20260806 11:40:47.114907 10710 ts_tablet_manager.cc:1939] T 2d3015cd13a54b819551ad6bb18925ff P 23fa000cf3544e019f01e58fd2e28604: Deleting consensus metadata
I20260806 11:40:47.115172 10711 ts_tablet_manager.cc:1939] T 2d3015cd13a54b819551ad6bb18925ff P 571412cc94054332be01c037c92139a3: Deleting consensus metadata
I20260806 11:40:47.116499 10183 catalog_manager.cc:4841] TS 571412cc94054332be01c037c92139a3 (127.21.14.195:34939): tablet 2d3015cd13a54b819551ad6bb18925ff (table test-table [id=07a22dc59a6d46ba8eaab173f66b287e]) successfully deleted
I20260806 11:40:47.116499 10180 catalog_manager.cc:4841] TS 23fa000cf3544e019f01e58fd2e28604 (127.21.14.194:36355): tablet 2d3015cd13a54b819551ad6bb18925ff (table test-table [id=07a22dc59a6d46ba8eaab173f66b287e]) successfully deleted
W20260806 11:40:47.117175 10183 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.14.193:39271 (error 108)
I20260806 11:40:47.117383 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 10395
I20260806 11:40:47.120584 10681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:47.120146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.193:39271 (local address 127.0.0.1:42192)
0806 11:40:47.120340 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0806 11:40:47.120450 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.193:39271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260806 11:40:47.120833 10183 catalog_manager.cc:4584] TS 482f411b88f248d69a2dbd5f6a486201 (127.21.14.193:39271): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2d3015cd13a54b819551ad6bb18925ff: Network error: Client connection negotiation failed: client connection to 127.21.14.193:39271: connect: Connection refused (error 111)
I20260806 11:40:47.126117 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 10533
I20260806 11:40:47.135386 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 10164
2026-08-06T11:40:47Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1124 ms)
[----------] 6 tests from EncryptionPolicyTest (11170 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786016169045816-21563-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 11:40:47 dist-test-slave-7c35 krb5kdc[10716](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 11:40:47 dist-test-slave-7c35 krb5kdc[10716](info): set up 2 sockets
Aug 06 11:40:47 dist-test-slave-7c35 krb5kdc[10716](info): commencing operation
krb5kdc: starting...
W20260806 11:40:48.262527 21563 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Aug 06 11:40:48 dist-test-slave-7c35 krb5kdc[10716](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016448, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T11:40:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T11:40:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786016169045816-21563-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786016169045816-21563-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 11:40:48.408016 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786016169045816-21563-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786016169045816-21563-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786016169045816-21563-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46341
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786016169045816-21563-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46341
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:40:48.530498 10729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:40:48.530827 10729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:40:48.530956 10729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:40:48.535239 10729 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 11:40:48.535305 10729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:40:48.535326 10729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:40:48.535342 10729 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 11:40:48.535398 10729 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 11:40:48.542148 10729 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786016169045816-21563-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786016169045816-21563-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46341
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786016169045816-21563-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46341
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10729
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786016169045816-21563-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:40:48.543529 10729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:40:48.544659 10729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:40:48.554037 10735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:40:48.554025 10734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:40:48.554282 10729 server_base.cc:1034] running on GCE node
W20260806 11:40:48.554056 10737 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:40:48.555089 10729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:40:48.556504 10729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:40:48.557704 10729 hybrid_clock.cc:648] HybridClock initialized: now 1786016448557672 us; error 47 us; skew 500 ppm
Aug 06 11:40:48 dist-test-slave-7c35 krb5kdc[10716](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016448, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:40:48.560761 10729 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 11:40:48.561874 10729 webserver.cc:460] Webserver started at http://127.0.0.1:46477/ using document root <none> and password file <none>
I20260806 11:40:48.562490 10729 fs_manager.cc:359] Metadata directory not provided
I20260806 11:40:48.562585 10729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:40:48.562829 10729 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:40:48.565104 10729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786016169045816-21563-0/minicluster-data/master-0/data/instance:
uuid: "0b9244bd17dd482dacf2e73ea4c75249"
format_stamp: "Formatted at 2026-08-06 11:40:48 on dist-test-slave-7c35"
server_key: "758d4da777dbb8eeafb883078659cf29"
server_key_iv: "218aa8e11d1319f3a35acf1e9816c374"
server_key_version: "encryptionkey@0"
I20260806 11:40:48.565763 10729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786016169045816-21563-0/minicluster-data/master-0/wal/instance:
uuid: "0b9244bd17dd482dacf2e73ea4c75249"
format_stamp: "Formatted at 2026-08-06 11:40:48 on dist-test-slave-7c35"
server_key: "758d4da777dbb8eeafb883078659cf29"
server_key_iv: "218aa8e11d1319f3a35acf1e9816c374"
server_key_version: "encryptionkey@0"
I20260806 11:40:48.569885 10729 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 11:40:48.572985 10744 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:48.574416 10729 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 11:40:48.574589 10729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786016169045816-21563-0/minicluster-data/master-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786016169045816-21563-0/minicluster-data/master-0/wal
uuid: "0b9244bd17dd482dacf2e73ea4c75249"
format_stamp: "Formatted at 2026-08-06 11:40:48 on dist-test-slave-7c35"
server_key: "758d4da777dbb8eeafb883078659cf29"
server_key_iv: "218aa8e11d1319f3a35acf1e9816c374"
server_key_version: "encryptionkey@0"
I20260806 11:40:48.574723 10729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786016169045816-21563-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786016169045816-21563-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786016169045816-21563-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:40:48.595737 10729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:40:48.599550 10729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:40:48.599799 10729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:40:48.608088 10729 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46341
I20260806 11:40:48.608165 10799 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46341 every 8 connection(s)
I20260806 11:40:48.609246 10729 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
I20260806 11:40:48.612493 10800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:40:48.616704 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 10729
I20260806 11:40:48.616894 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786016169045816-21563-0/minicluster-data/master-0/wal/instance
I20260806 11:40:48.617210 21563 external_mini_cluster.cc:1428] Setting key 5fa7678d5df192c48592a92dac73e503
I20260806 11:40:48.619606 10800 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b9244bd17dd482dacf2e73ea4c75249: Bootstrap starting.
I20260806 11:40:48.622471 10800 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0b9244bd17dd482dacf2e73ea4c75249: Neither blocks nor log segments found. Creating new log.
I20260806 11:40:48.623493 10800 log.cc:824] T 00000000000000000000000000000000 P 0b9244bd17dd482dacf2e73ea4c75249: Log is configured to *not* fsync() on all Append() calls
I20260806 11:40:48.625965 10800 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b9244bd17dd482dacf2e73ea4c75249: No bootstrap required, opened a new log
Aug 06 11:40:48 dist-test-slave-7c35 krb5kdc[10716](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016448, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 11:40:48.630286 10800 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0b9244bd17dd482dacf2e73ea4c75249 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b9244bd17dd482dacf2e73ea4c75249" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46341 } }
I20260806 11:40:48.630564 10800 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0b9244bd17dd482dacf2e73ea4c75249 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:40:48.630601 10800 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0b9244bd17dd482dacf2e73ea4c75249 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b9244bd17dd482dacf2e73ea4c75249, State: Initialized, Role: FOLLOWER
I20260806 11:40:48.631177 10800 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0b9244bd17dd482dacf2e73ea4c75249 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b9244bd17dd482dacf2e73ea4c75249" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46341 } }
I20260806 11:40:48.631326 10800 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0b9244bd17dd482dacf2e73ea4c75249 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 11:40:48.631418 10800 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0b9244bd17dd482dacf2e73ea4c75249 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 11:40:48.631557 10800 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0b9244bd17dd482dacf2e73ea4c75249 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:40:48.632642 10800 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0b9244bd17dd482dacf2e73ea4c75249 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b9244bd17dd482dacf2e73ea4c75249" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46341 } }
I20260806 11:40:48.633051 10800 leader_election.cc:302] T 00000000000000000000000000000000 P 0b9244bd17dd482dacf2e73ea4c75249 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0b9244bd17dd482dacf2e73ea4c75249; no voters:
I20260806 11:40:48.633481 10800 leader_election.cc:288] T 00000000000000000000000000000000 P 0b9244bd17dd482dacf2e73ea4c75249 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 11:40:48.633589 10806 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0b9244bd17dd482dacf2e73ea4c75249 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:40:48.633873 10806 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0b9244bd17dd482dacf2e73ea4c75249 [term 1 LEADER]: Becoming Leader. State: Replica: 0b9244bd17dd482dacf2e73ea4c75249, State: Running, Role: LEADER
I20260806 11:40:48.634294 10806 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0b9244bd17dd482dacf2e73ea4c75249 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b9244bd17dd482dacf2e73ea4c75249" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46341 } }
I20260806 11:40:48.634712 10804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:48.619024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50217 (local address 127.0.0.1:46341)
0806 11:40:48.619533 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:48.619552 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:48.619591 (+ 39us) server_negotiation.cc:407] Connection header received
0806 11:40:48.620333 (+ 742us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:48.620352 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:48.620748 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:48.621063 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:40:48.622048 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:48.623297 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:48.624484 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:48.625003 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:48.625192 (+ 189us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:48.628287 (+ 3095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:40:48.628293 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:40:48.628311 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:40:48.628344 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:40:48.630837 (+ 2493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:40:48.631553 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:40:48.631565 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:40:48.631572 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:40:48.631741 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:40:48.632128 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:40:48.632132 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:40:48.632134 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:40:48.632601 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:40:48.632890 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:48.633253 (+ 363us) server_negotiation.cc:299] Negotiation successful
0806 11:40:48.633568 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":173,"threads_started":1}
I20260806 11:40:48.634994 10800 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0b9244bd17dd482dacf2e73ea4c75249 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 11:40:48.635669 10806 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b9244bd17dd482dacf2e73ea4c75249 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b9244bd17dd482dacf2e73ea4c75249. Latest consensus state: current_term: 1 leader_uuid: "0b9244bd17dd482dacf2e73ea4c75249" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b9244bd17dd482dacf2e73ea4c75249" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46341 } } }
I20260806 11:40:48.637385 10806 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b9244bd17dd482dacf2e73ea4c75249 [sys.catalog]: This master's current role is: LEADER
I20260806 11:40:48.637256 10807 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b9244bd17dd482dacf2e73ea4c75249 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0b9244bd17dd482dacf2e73ea4c75249" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b9244bd17dd482dacf2e73ea4c75249" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46341 } } }
I20260806 11:40:48.637837 10807 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b9244bd17dd482dacf2e73ea4c75249 [sys.catalog]: This master's current role is: LEADER
I20260806 11:40:48.638406 21563 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260806 11:40:48.640842 10820 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0b9244bd17dd482dacf2e73ea4c75249: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 11:40:48.641005 10820 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 11:40:48.641103 10821 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 11:40:48.642099 10821 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 11:40:48.649288 10821 catalog_manager.cc:1347] Generated new cluster ID: 8460119dbf2c4681ac58d6abfcc12951
I20260806 11:40:48.649367 10821 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 11:40:48.650218 10804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:48.640705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56243 (local address 127.0.0.1:46341)
0806 11:40:48.640827 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:48.640832 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:48.641147 (+ 315us) server_negotiation.cc:407] Connection header received
0806 11:40:48.641359 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:48.641363 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:48.641415 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:48.641508 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:40:48.642416 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:48.643263 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:48.644328 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:48.644580 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:48.644693 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:48.646171 (+ 1478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:40:48.646175 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:40:48.646178 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:40:48.646212 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:40:48.648067 (+ 1855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:40:48.648703 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:40:48.648707 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:40:48.648709 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:40:48.648805 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:40:48.649216 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:40:48.649220 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:40:48.649222 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:40:48.649461 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:40:48.649622 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:48.649930 (+ 308us) server_negotiation.cc:299] Negotiation successful
0806 11:40:48.650055 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 11:40:48.670514 10821 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 11:40:48.671967 10821 catalog_manager.cc:1504] Loading token signing keys...
I20260806 11:40:48.677892 10821 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0b9244bd17dd482dacf2e73ea4c75249: Generated new TSK 0
I20260806 11:40:48.678799 10821 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 11:40:48.685597 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 10729
W20260806 11:40:48.693900 10714 connection.cc:535] client connection to 127.0.0.1:46341 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46341): Connection reset by peer (error 104)
2026-08-06T11:40:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1547 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786016169045816-21563-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 11:40:48 dist-test-slave-7c35 krb5kdc[10834](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 11:40:48 dist-test-slave-7c35 krb5kdc[10834](info): set up 2 sockets
Aug 06 11:40:48 dist-test-slave-7c35 krb5kdc[10834](info): commencing operation
krb5kdc: starting...
W20260806 11:40:49.762034 21563 mini_kdc.cc:121] Time spent starting KDC: real 1.053s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Aug 06 11:40:49 dist-test-slave-7c35 krb5kdc[10834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016449, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T11:40:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T11:40:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786016169045816-21563-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786016169045816-21563-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 11:40:49.902118 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786016169045816-21563-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786016169045816-21563-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786016169045816-21563-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33299
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786016169045816-21563-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33299
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:40:50.020366 10847 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:40:50.020713 10847 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:40:50.020807 10847 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:40:50.025130 10847 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 11:40:50.025235 10847 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:40:50.025274 10847 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:40:50.025314 10847 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 11:40:50.025350 10847 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 11:40:50.031657 10847 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786016169045816-21563-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786016169045816-21563-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33299
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786016169045816-21563-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33299
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10847
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786016169045816-21563-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:40:50.032902 10847 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:40:50.034039 10847 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:40:50.043345 10855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:40:50.043558 10852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:40:50.043345 10853 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:40:50.043757 10847 server_base.cc:1034] running on GCE node
I20260806 11:40:50.044384 10847 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:40:50.045619 10847 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:40:50.046820 10847 hybrid_clock.cc:648] HybridClock initialized: now 1786016450046805 us; error 28 us; skew 500 ppm
Aug 06 11:40:50 dist-test-slave-7c35 krb5kdc[10834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016450, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:40:50.049908 10847 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 11:40:50.050837 10847 webserver.cc:460] Webserver started at http://127.0.0.1:42813/ using document root <none> and password file <none>
I20260806 11:40:50.051451 10847 fs_manager.cc:359] Metadata directory not provided
I20260806 11:40:50.051520 10847 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:40:50.051690 10847 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:40:50.053763 10847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786016169045816-21563-0/minicluster-data/master-0/data/instance:
uuid: "ddfebfed16704d35af7f12ea79b9b0d3"
format_stamp: "Formatted at 2026-08-06 11:40:50 on dist-test-slave-7c35"
server_key: "b15fd161fcb8fcb9887f50ccf6445703"
server_key_iv: "7324aa426ce7f39e800f4a7064833d18"
server_key_version: "encryptionkey@0"
I20260806 11:40:50.054373 10847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786016169045816-21563-0/minicluster-data/master-0/wal/instance:
uuid: "ddfebfed16704d35af7f12ea79b9b0d3"
format_stamp: "Formatted at 2026-08-06 11:40:50 on dist-test-slave-7c35"
server_key: "b15fd161fcb8fcb9887f50ccf6445703"
server_key_iv: "7324aa426ce7f39e800f4a7064833d18"
server_key_version: "encryptionkey@0"
I20260806 11:40:50.058390 10847 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260806 11:40:50.061232 10863 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:50.062570 10847 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 11:40:50.062712 10847 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786016169045816-21563-0/minicluster-data/master-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786016169045816-21563-0/minicluster-data/master-0/wal
uuid: "ddfebfed16704d35af7f12ea79b9b0d3"
format_stamp: "Formatted at 2026-08-06 11:40:50 on dist-test-slave-7c35"
server_key: "b15fd161fcb8fcb9887f50ccf6445703"
server_key_iv: "7324aa426ce7f39e800f4a7064833d18"
server_key_version: "encryptionkey@0"
I20260806 11:40:50.062829 10847 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786016169045816-21563-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786016169045816-21563-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786016169045816-21563-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:40:50.079123 10847 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:40:50.082820 10847 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:40:50.083093 10847 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:40:50.091208 10847 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33299
I20260806 11:40:50.091301 10917 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33299 every 8 connection(s)
I20260806 11:40:50.092322 10847 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
I20260806 11:40:50.095423 10918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:40:50.100168 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 10847
I20260806 11:40:50.100298 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786016169045816-21563-0/minicluster-data/master-0/wal/instance
I20260806 11:40:50.100654 21563 external_mini_cluster.cc:1428] Setting key 9b75fb4bd692d693a2557ae6dc6e7d29
I20260806 11:40:50.101662 10918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ddfebfed16704d35af7f12ea79b9b0d3: Bootstrap starting.
I20260806 11:40:50.104380 10918 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ddfebfed16704d35af7f12ea79b9b0d3: Neither blocks nor log segments found. Creating new log.
I20260806 11:40:50.105199 10918 log.cc:824] T 00000000000000000000000000000000 P ddfebfed16704d35af7f12ea79b9b0d3: Log is configured to *not* fsync() on all Append() calls
I20260806 11:40:50.107332 10918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ddfebfed16704d35af7f12ea79b9b0d3: No bootstrap required, opened a new log
I20260806 11:40:50.110807 10921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:50.102362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33319 (local address 127.0.0.1:33299)
0806 11:40:50.102926 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:50.102944 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:50.102986 (+ 42us) server_negotiation.cc:407] Connection header received
0806 11:40:50.103773 (+ 787us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:50.103793 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:50.104192 (+ 399us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:50.104582 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:40:50.105043 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:50.106065 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:50.107060 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:50.107557 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:50.107727 (+ 170us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:50.108749 (+ 1022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:40:50.108755 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:40:50.108769 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 11:40:50.109186 (+ 417us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:40:50.109249 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 11:40:50.109273 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:40:50.109423 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:50.109616 (+ 193us) server_negotiation.cc:299] Negotiation successful
0806 11:40:50.109767 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":160,"threads_started":1}
I20260806 11:40:50.111572 10918 raft_consensus.cc:348] T 00000000000000000000000000000000 P ddfebfed16704d35af7f12ea79b9b0d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddfebfed16704d35af7f12ea79b9b0d3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33299 } }
I20260806 11:40:50.111843 10918 raft_consensus.cc:374] T 00000000000000000000000000000000 P ddfebfed16704d35af7f12ea79b9b0d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:40:50.111919 10918 raft_consensus.cc:729] T 00000000000000000000000000000000 P ddfebfed16704d35af7f12ea79b9b0d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddfebfed16704d35af7f12ea79b9b0d3, State: Initialized, Role: FOLLOWER
I20260806 11:40:50.112432 10918 consensus_queue.cc:260] T 00000000000000000000000000000000 P ddfebfed16704d35af7f12ea79b9b0d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddfebfed16704d35af7f12ea79b9b0d3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33299 } }
I20260806 11:40:50.112617 10918 raft_consensus.cc:388] T 00000000000000000000000000000000 P ddfebfed16704d35af7f12ea79b9b0d3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 11:40:50.112692 10918 raft_consensus.cc:482] T 00000000000000000000000000000000 P ddfebfed16704d35af7f12ea79b9b0d3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 11:40:50.112792 10918 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ddfebfed16704d35af7f12ea79b9b0d3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:40:50.114063 10918 raft_consensus.cc:504] T 00000000000000000000000000000000 P ddfebfed16704d35af7f12ea79b9b0d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddfebfed16704d35af7f12ea79b9b0d3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33299 } }
I20260806 11:40:50.114548 21563 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 11:40:50.114516 10918 leader_election.cc:302] T 00000000000000000000000000000000 P ddfebfed16704d35af7f12ea79b9b0d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ddfebfed16704d35af7f12ea79b9b0d3; no voters:
I20260806 11:40:50.115120 10918 leader_election.cc:288] T 00000000000000000000000000000000 P ddfebfed16704d35af7f12ea79b9b0d3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 11:40:50.115294 10923 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ddfebfed16704d35af7f12ea79b9b0d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:40:50.115578 10923 raft_consensus.cc:686] T 00000000000000000000000000000000 P ddfebfed16704d35af7f12ea79b9b0d3 [term 1 LEADER]: Becoming Leader. State: Replica: ddfebfed16704d35af7f12ea79b9b0d3, State: Running, Role: LEADER
I20260806 11:40:50.116014 10923 consensus_queue.cc:237] T 00000000000000000000000000000000 P ddfebfed16704d35af7f12ea79b9b0d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddfebfed16704d35af7f12ea79b9b0d3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33299 } }
I20260806 11:40:50.116892 10918 sys_catalog.cc:564] T 00000000000000000000000000000000 P ddfebfed16704d35af7f12ea79b9b0d3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 11:40:50.118189 10924 sys_catalog.cc:455] T 00000000000000000000000000000000 P ddfebfed16704d35af7f12ea79b9b0d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ddfebfed16704d35af7f12ea79b9b0d3. Latest consensus state: current_term: 1 leader_uuid: "ddfebfed16704d35af7f12ea79b9b0d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddfebfed16704d35af7f12ea79b9b0d3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33299 } } }
I20260806 11:40:50.118356 10924 sys_catalog.cc:458] T 00000000000000000000000000000000 P ddfebfed16704d35af7f12ea79b9b0d3 [sys.catalog]: This master's current role is: LEADER
I20260806 11:40:50.118788 10938 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 11:40:50.118732 10925 sys_catalog.cc:455] T 00000000000000000000000000000000 P ddfebfed16704d35af7f12ea79b9b0d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ddfebfed16704d35af7f12ea79b9b0d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddfebfed16704d35af7f12ea79b9b0d3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33299 } } }
I20260806 11:40:50.118826 10925 sys_catalog.cc:458] T 00000000000000000000000000000000 P ddfebfed16704d35af7f12ea79b9b0d3 [sys.catalog]: This master's current role is: LEADER
I20260806 11:40:50.122329 10938 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 11:40:50.123039 10921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:50.117061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50203 (local address 127.0.0.1:33299)
0806 11:40:50.117185 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:50.117189 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:50.117552 (+ 363us) server_negotiation.cc:407] Connection header received
0806 11:40:50.117781 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:50.117787 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:50.117856 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:50.117975 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:40:50.118390 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:50.119291 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:50.120574 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:50.120947 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:50.121062 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:50.122187 (+ 1125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:40:50.122191 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:40:50.122194 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 11:40:50.122210 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:40:50.122241 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 11:40:50.122256 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:40:50.122357 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:50.122820 (+ 463us) server_negotiation.cc:299] Negotiation successful
0806 11:40:50.122892 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 11:40:50.129993 10938 catalog_manager.cc:1347] Generated new cluster ID: 0a06de45fe1b4aec91043ee1d86e6beb
I20260806 11:40:50.130123 10938 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 11:40:50.145540 10938 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 11:40:50.146610 10938 catalog_manager.cc:1504] Loading token signing keys...
I20260806 11:40:50.157042 10938 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ddfebfed16704d35af7f12ea79b9b0d3: Generated new TSK 0
I20260806 11:40:50.157912 10938 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 11:40:50.163690 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 10847
W20260806 11:40:50.172271 10830 connection.cc:535] client connection to 127.0.0.1:33299 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33299): Connection reset by peer (error 104)
2026-08-06T11:40:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1478 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786016169045816-21563-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 11:40:50 dist-test-slave-7c35 krb5kdc[10951](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 11:40:50 dist-test-slave-7c35 krb5kdc[10951](info): set up 2 sockets
Aug 06 11:40:50 dist-test-slave-7c35 krb5kdc[10951](info): commencing operation
krb5kdc: starting...
W20260806 11:40:51.257512 21563 mini_kdc.cc:121] Time spent starting KDC: real 1.070s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Aug 06 11:40:51 dist-test-slave-7c35 krb5kdc[10951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016451, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T11:40:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T11:40:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786016169045816-21563-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786016169045816-21563-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 11:40:51.399150 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786016169045816-21563-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786016169045816-21563-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786016169045816-21563-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39149
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786016169045816-21563-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39149
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:40:51.519380 10964 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:40:51.519752 10964 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:40:51.519827 10964 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:40:51.524098 10964 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 11:40:51.524219 10964 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:40:51.524256 10964 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:40:51.524293 10964 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 11:40:51.524325 10964 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 11:40:51.530670 10964 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786016169045816-21563-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786016169045816-21563-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39149
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786016169045816-21563-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39149
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10964
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786016169045816-21563-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:40:51.532015 10964 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:40:51.533224 10964 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:40:51.542629 10970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:40:51.542616 10969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:40:51.542687 10972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:40:51.542836 10964 server_base.cc:1034] running on GCE node
I20260806 11:40:51.543686 10964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:40:51.544934 10964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:40:51.546132 10964 hybrid_clock.cc:648] HybridClock initialized: now 1786016451546108 us; error 50 us; skew 500 ppm
Aug 06 11:40:51 dist-test-slave-7c35 krb5kdc[10951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016451, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:40:51.555841 10964 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 11:40:51.557061 10964 webserver.cc:460] Webserver started at http://127.0.0.1:34857/ using document root <none> and password file <none>
I20260806 11:40:51.557643 10964 fs_manager.cc:359] Metadata directory not provided
I20260806 11:40:51.557708 10964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:40:51.557886 10964 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:40:51.560184 10964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786016169045816-21563-0/minicluster-data/master-0/data/instance:
uuid: "c613c6545bc34176900d630028bfc88d"
format_stamp: "Formatted at 2026-08-06 11:40:51 on dist-test-slave-7c35"
server_key: "cf760cdc9383319cbf87c7f0c2cba7db"
server_key_iv: "03275fa4e4e58760f1d4aa60027ed964"
server_key_version: "encryptionkey@0"
I20260806 11:40:51.560755 10964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786016169045816-21563-0/minicluster-data/master-0/wal/instance:
uuid: "c613c6545bc34176900d630028bfc88d"
format_stamp: "Formatted at 2026-08-06 11:40:51 on dist-test-slave-7c35"
server_key: "cf760cdc9383319cbf87c7f0c2cba7db"
server_key_iv: "03275fa4e4e58760f1d4aa60027ed964"
server_key_version: "encryptionkey@0"
I20260806 11:40:51.564913 10964 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260806 11:40:51.567731 10980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:51.569175 10964 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 11:40:51.569416 10964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786016169045816-21563-0/minicluster-data/master-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786016169045816-21563-0/minicluster-data/master-0/wal
uuid: "c613c6545bc34176900d630028bfc88d"
format_stamp: "Formatted at 2026-08-06 11:40:51 on dist-test-slave-7c35"
server_key: "cf760cdc9383319cbf87c7f0c2cba7db"
server_key_iv: "03275fa4e4e58760f1d4aa60027ed964"
server_key_version: "encryptionkey@0"
I20260806 11:40:51.569541 10964 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786016169045816-21563-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786016169045816-21563-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786016169045816-21563-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:40:51.574389 10964 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:40:51.577574 10964 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:40:51.577776 10964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:40:51.585237 10964 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39149
I20260806 11:40:51.585258 11034 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39149 every 8 connection(s)
I20260806 11:40:51.586241 10964 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
I20260806 11:40:51.587133 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 10964
I20260806 11:40:51.587278 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786016169045816-21563-0/minicluster-data/master-0/wal/instance
I20260806 11:40:51.587574 21563 external_mini_cluster.cc:1428] Setting key e55c26f6b9a91bb695adeddae8e18df1
I20260806 11:40:51.589856 11035 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:40:51.593485 11039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:51.588929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55603 (local address 127.0.0.1:39149)
0806 11:40:51.589375 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:51.589396 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:51.589442 (+ 46us) server_negotiation.cc:407] Connection header received
0806 11:40:51.590339 (+ 897us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:51.590363 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:51.590738 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:51.591168 (+ 430us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:40:51.591947 (+ 779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:40:51.591953 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:40:51.591970 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 11:40:51.591986 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:40:51.592068 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 11:40:51.592095 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:40:51.592250 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:51.592325 (+ 75us) server_negotiation.cc:299] Negotiation successful
0806 11:40:51.592488 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":122,"threads_started":1}
I20260806 11:40:51.596971 21563 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 11:40:51.597534 11035 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c613c6545bc34176900d630028bfc88d: Bootstrap starting.
I20260806 11:40:51.600445 11035 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c613c6545bc34176900d630028bfc88d: Neither blocks nor log segments found. Creating new log.
I20260806 11:40:51.601140 11039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:51.599277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49803 (local address 127.0.0.1:39149)
0806 11:40:51.599428 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:51.599433 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:51.599685 (+ 252us) server_negotiation.cc:407] Connection header received
0806 11:40:51.599802 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:51.599805 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:51.599842 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:51.599923 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:40:51.600703 (+ 780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:40:51.600706 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:40:51.600708 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 11:40:51.600719 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:40:51.600750 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 11:40:51.600761 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:40:51.600838 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:51.600954 (+ 116us) server_negotiation.cc:299] Negotiation successful
0806 11:40:51.600991 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 11:40:51.601541 11035 log.cc:824] T 00000000000000000000000000000000 P c613c6545bc34176900d630028bfc88d: Log is configured to *not* fsync() on all Append() calls
I20260806 11:40:51.603824 11035 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c613c6545bc34176900d630028bfc88d: No bootstrap required, opened a new log
I20260806 11:40:51.607690 11035 raft_consensus.cc:348] T 00000000000000000000000000000000 P c613c6545bc34176900d630028bfc88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c613c6545bc34176900d630028bfc88d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39149 } }
I20260806 11:40:51.607985 11035 raft_consensus.cc:374] T 00000000000000000000000000000000 P c613c6545bc34176900d630028bfc88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:40:51.608071 11035 raft_consensus.cc:729] T 00000000000000000000000000000000 P c613c6545bc34176900d630028bfc88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c613c6545bc34176900d630028bfc88d, State: Initialized, Role: FOLLOWER
I20260806 11:40:51.608587 11035 consensus_queue.cc:260] T 00000000000000000000000000000000 P c613c6545bc34176900d630028bfc88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c613c6545bc34176900d630028bfc88d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39149 } }
I20260806 11:40:51.608753 11035 raft_consensus.cc:388] T 00000000000000000000000000000000 P c613c6545bc34176900d630028bfc88d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 11:40:51.608853 11035 raft_consensus.cc:482] T 00000000000000000000000000000000 P c613c6545bc34176900d630028bfc88d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 11:40:51.608966 11035 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c613c6545bc34176900d630028bfc88d [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:40:51.610057 11035 raft_consensus.cc:504] T 00000000000000000000000000000000 P c613c6545bc34176900d630028bfc88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c613c6545bc34176900d630028bfc88d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39149 } }
I20260806 11:40:51.610468 11035 leader_election.cc:302] T 00000000000000000000000000000000 P c613c6545bc34176900d630028bfc88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c613c6545bc34176900d630028bfc88d; no voters:
I20260806 11:40:51.610898 11035 leader_election.cc:288] T 00000000000000000000000000000000 P c613c6545bc34176900d630028bfc88d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 11:40:51.611073 11046 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c613c6545bc34176900d630028bfc88d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:40:51.611357 11046 raft_consensus.cc:686] T 00000000000000000000000000000000 P c613c6545bc34176900d630028bfc88d [term 1 LEADER]: Becoming Leader. State: Replica: c613c6545bc34176900d630028bfc88d, State: Running, Role: LEADER
I20260806 11:40:51.611800 11046 consensus_queue.cc:237] T 00000000000000000000000000000000 P c613c6545bc34176900d630028bfc88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c613c6545bc34176900d630028bfc88d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39149 } }
I20260806 11:40:51.612460 11035 sys_catalog.cc:564] T 00000000000000000000000000000000 P c613c6545bc34176900d630028bfc88d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 11:40:51.613683 11047 sys_catalog.cc:455] T 00000000000000000000000000000000 P c613c6545bc34176900d630028bfc88d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c613c6545bc34176900d630028bfc88d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c613c6545bc34176900d630028bfc88d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39149 } } }
I20260806 11:40:51.613859 11047 sys_catalog.cc:458] T 00000000000000000000000000000000 P c613c6545bc34176900d630028bfc88d [sys.catalog]: This master's current role is: LEADER
I20260806 11:40:51.614229 11048 sys_catalog.cc:455] T 00000000000000000000000000000000 P c613c6545bc34176900d630028bfc88d [sys.catalog]: SysCatalogTable state changed. Reason: New leader c613c6545bc34176900d630028bfc88d. Latest consensus state: current_term: 1 leader_uuid: "c613c6545bc34176900d630028bfc88d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c613c6545bc34176900d630028bfc88d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39149 } } }
I20260806 11:40:51.614401 11048 sys_catalog.cc:458] T 00000000000000000000000000000000 P c613c6545bc34176900d630028bfc88d [sys.catalog]: This master's current role is: LEADER
I20260806 11:40:51.615223 11055 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 11:40:51.619868 11055 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 11:40:51.627497 11055 catalog_manager.cc:1347] Generated new cluster ID: 5bd72fe270924dfb80afddbd76c22371
I20260806 11:40:51.627595 11055 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 11:40:51.656229 11055 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 11:40:51.656647 11055 catalog_manager.cc:1504] Loading token signing keys...
I20260806 11:40:51.669973 11055 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c613c6545bc34176900d630028bfc88d: Generated new TSK 0
I20260806 11:40:51.670795 11055 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 11:40:51.686635 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 10964
2026-08-06T11:40:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1522 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786016169045816-21563-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 11:40:51 dist-test-slave-7c35 krb5kdc[11068](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 11:40:51 dist-test-slave-7c35 krb5kdc[11068](info): set up 2 sockets
Aug 06 11:40:51 dist-test-slave-7c35 krb5kdc[11068](info): commencing operation
krb5kdc: starting...
W20260806 11:40:52.766466 21563 mini_kdc.cc:121] Time spent starting KDC: real 1.057s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Aug 06 11:40:52 dist-test-slave-7c35 krb5kdc[11068](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016452, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T11:40:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T11:40:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786016169045816-21563-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786016169045816-21563-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 11:40:52.907353 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786016169045816-21563-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786016169045816-21563-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786016169045816-21563-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44383
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786016169045816-21563-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44383
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:40:53.027433 11081 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:40:53.027760 11081 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:40:53.027835 11081 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:40:53.032127 11081 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 11:40:53.032233 11081 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:40:53.032272 11081 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:40:53.032325 11081 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 11:40:53.032361 11081 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 11:40:53.038559 11081 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786016169045816-21563-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786016169045816-21563-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44383
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786016169045816-21563-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44383
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11081
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786016169045816-21563-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:40:53.039801 11081 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:40:53.040916 11081 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:40:53.050410 11088 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:40:53.050463 11081 server_base.cc:1034] running on GCE node
W20260806 11:40:53.050396 11087 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:40:53.050369 11090 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:40:53.051328 11081 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:40:53.052682 11081 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:40:53.053891 11081 hybrid_clock.cc:648] HybridClock initialized: now 1786016453053849 us; error 52 us; skew 500 ppm
Aug 06 11:40:53 dist-test-slave-7c35 krb5kdc[11068](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016453, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:40:53.057098 11081 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 11:40:53.058205 11081 webserver.cc:460] Webserver started at http://127.0.0.1:37809/ using document root <none> and password file <none>
I20260806 11:40:53.058949 11081 fs_manager.cc:359] Metadata directory not provided
I20260806 11:40:53.059056 11081 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:40:53.059319 11081 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:40:53.061691 11081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786016169045816-21563-0/minicluster-data/master-0/data/instance:
uuid: "2343dd471b86426ea897c3927346a927"
format_stamp: "Formatted at 2026-08-06 11:40:53 on dist-test-slave-7c35"
server_key: "3dee6ec8d64c9571ae3e4d1aaea78755"
server_key_iv: "2eb7e213533fa3056490a907f97a0403"
server_key_version: "encryptionkey@0"
I20260806 11:40:53.062357 11081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786016169045816-21563-0/minicluster-data/master-0/wal/instance:
uuid: "2343dd471b86426ea897c3927346a927"
format_stamp: "Formatted at 2026-08-06 11:40:53 on dist-test-slave-7c35"
server_key: "3dee6ec8d64c9571ae3e4d1aaea78755"
server_key_iv: "2eb7e213533fa3056490a907f97a0403"
server_key_version: "encryptionkey@0"
I20260806 11:40:53.066826 11081 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260806 11:40:53.069789 11097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:53.071376 11081 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 11:40:53.071552 11081 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786016169045816-21563-0/minicluster-data/master-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786016169045816-21563-0/minicluster-data/master-0/wal
uuid: "2343dd471b86426ea897c3927346a927"
format_stamp: "Formatted at 2026-08-06 11:40:53 on dist-test-slave-7c35"
server_key: "3dee6ec8d64c9571ae3e4d1aaea78755"
server_key_iv: "2eb7e213533fa3056490a907f97a0403"
server_key_version: "encryptionkey@0"
I20260806 11:40:53.071691 11081 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786016169045816-21563-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786016169045816-21563-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786016169045816-21563-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:40:53.087503 11081 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:40:53.091477 11081 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:40:53.091725 11081 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:40:53.099556 11081 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44383
I20260806 11:40:53.099573 11150 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44383 every 8 connection(s)
I20260806 11:40:53.100626 11081 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
I20260806 11:40:53.103708 11151 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:40:53.106508 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 11081
I20260806 11:40:53.106674 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786016169045816-21563-0/minicluster-data/master-0/wal/instance
I20260806 11:40:53.107050 21563 external_mini_cluster.cc:1428] Setting key 17c444e2fc66bf5b84146730848dad7f
I20260806 11:40:53.110898 11151 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2343dd471b86426ea897c3927346a927: Bootstrap starting.
I20260806 11:40:53.113929 11151 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2343dd471b86426ea897c3927346a927: Neither blocks nor log segments found. Creating new log.
I20260806 11:40:53.115039 11151 log.cc:824] T 00000000000000000000000000000000 P 2343dd471b86426ea897c3927346a927: Log is configured to *not* fsync() on all Append() calls
I20260806 11:40:53.117841 11151 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2343dd471b86426ea897c3927346a927: No bootstrap required, opened a new log
Aug 06 11:40:53 dist-test-slave-7c35 krb5kdc[11068](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016452, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 11:40:53.121924 11151 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2343dd471b86426ea897c3927346a927 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2343dd471b86426ea897c3927346a927" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44383 } }
I20260806 11:40:53.122243 11151 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2343dd471b86426ea897c3927346a927 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:40:53.122308 11151 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2343dd471b86426ea897c3927346a927 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2343dd471b86426ea897c3927346a927, State: Initialized, Role: FOLLOWER
I20260806 11:40:53.122917 11151 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2343dd471b86426ea897c3927346a927 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2343dd471b86426ea897c3927346a927" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44383 } }
I20260806 11:40:53.123116 11151 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2343dd471b86426ea897c3927346a927 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 11:40:53.123225 11151 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2343dd471b86426ea897c3927346a927 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 11:40:53.123345 11151 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2343dd471b86426ea897c3927346a927 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:40:53.123814 11154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:53.108923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55585 (local address 127.0.0.1:44383)
0806 11:40:53.109511 (+ 588us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:53.109530 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:53.109572 (+ 42us) server_negotiation.cc:407] Connection header received
0806 11:40:53.110300 (+ 728us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:53.110319 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:53.110708 (+ 389us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:53.111172 (+ 464us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:40:53.112198 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:53.113245 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:53.114614 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:53.115276 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:53.115415 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:53.118568 (+ 3153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:40:53.118574 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:40:53.118589 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:40:53.118619 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:40:53.120487 (+ 1868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:40:53.120990 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:40:53.120995 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:40:53.121000 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:40:53.121123 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:40:53.121425 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:40:53.121429 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:40:53.121430 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:40:53.121879 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:40:53.122113 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:53.122405 (+ 292us) server_negotiation.cc:299] Negotiation successful
0806 11:40:53.122748 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":164,"threads_started":1}
I20260806 11:40:53.124866 11151 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2343dd471b86426ea897c3927346a927 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2343dd471b86426ea897c3927346a927" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44383 } }
I20260806 11:40:53.125329 11151 leader_election.cc:302] T 00000000000000000000000000000000 P 2343dd471b86426ea897c3927346a927 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2343dd471b86426ea897c3927346a927; no voters:
I20260806 11:40:53.125758 11151 leader_election.cc:288] T 00000000000000000000000000000000 P 2343dd471b86426ea897c3927346a927 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 11:40:53.125900 11156 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2343dd471b86426ea897c3927346a927 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:40:53.126156 11156 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2343dd471b86426ea897c3927346a927 [term 1 LEADER]: Becoming Leader. State: Replica: 2343dd471b86426ea897c3927346a927, State: Running, Role: LEADER
I20260806 11:40:53.126536 11156 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2343dd471b86426ea897c3927346a927 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2343dd471b86426ea897c3927346a927" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44383 } }
I20260806 11:40:53.127349 11151 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2343dd471b86426ea897c3927346a927 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 11:40:53.127985 21563 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 11:40:53.128719 11157 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2343dd471b86426ea897c3927346a927 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2343dd471b86426ea897c3927346a927" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2343dd471b86426ea897c3927346a927" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44383 } } }
I20260806 11:40:53.128893 11157 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2343dd471b86426ea897c3927346a927 [sys.catalog]: This master's current role is: LEADER
I20260806 11:40:53.128732 11159 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2343dd471b86426ea897c3927346a927 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2343dd471b86426ea897c3927346a927. Latest consensus state: current_term: 1 leader_uuid: "2343dd471b86426ea897c3927346a927" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2343dd471b86426ea897c3927346a927" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44383 } } }
I20260806 11:40:53.129021 11159 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2343dd471b86426ea897c3927346a927 [sys.catalog]: This master's current role is: LEADER
I20260806 11:40:53.129400 11167 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 11:40:53.133090 11167 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 11:40:53.141538 11154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:53.130682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34951 (local address 127.0.0.1:44383)
0806 11:40:53.130809 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:53.130814 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:53.131169 (+ 355us) server_negotiation.cc:407] Connection header received
0806 11:40:53.131381 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:53.131384 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:53.131440 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:53.131529 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:40:53.132947 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:53.133758 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:53.135022 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:53.135431 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:53.135572 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:53.137504 (+ 1932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:40:53.137508 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:40:53.137511 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:40:53.137547 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:40:53.139578 (+ 2031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:40:53.140175 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:40:53.140179 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:40:53.140181 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:40:53.140279 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:40:53.140580 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:40:53.140584 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:40:53.140586 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:40:53.140799 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:40:53.140936 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:53.141193 (+ 257us) server_negotiation.cc:299] Negotiation successful
0806 11:40:53.141332 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 11:40:53.141662 11167 catalog_manager.cc:1347] Generated new cluster ID: 743fcdff09544f78b6e330d855be2324
I20260806 11:40:53.141731 11167 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 11:40:53.165644 11167 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 11:40:53.167107 11167 catalog_manager.cc:1504] Loading token signing keys...
I20260806 11:40:53.172662 11167 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2343dd471b86426ea897c3927346a927: Generated new TSK 0
I20260806 11:40:53.173471 11167 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 11:40:53.179903 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 11081
2026-08-06T11:40:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1492 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786016169045816-21563-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 11:40:53 dist-test-slave-7c35 krb5kdc[11184](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 11:40:53 dist-test-slave-7c35 krb5kdc[11184](info): set up 2 sockets
Aug 06 11:40:53 dist-test-slave-7c35 krb5kdc[11184](info): commencing operation
krb5kdc: starting...
W20260806 11:40:54.271025 21563 mini_kdc.cc:121] Time spent starting KDC: real 1.068s user 0.008s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Aug 06 11:40:54 dist-test-slave-7c35 krb5kdc[11184](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016454, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T11:40:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T11:40:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786016169045816-21563-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786016169045816-21563-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 11:40:54.420161 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786016169045816-21563-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786016169045816-21563-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786016169045816-21563-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37937
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786016169045816-21563-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37937
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:40:54.538136 11198 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:40:54.538522 11198 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:40:54.538620 11198 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:40:54.543699 11198 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 11:40:54.543823 11198 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:40:54.543864 11198 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:40:54.543906 11198 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 11:40:54.543951 11198 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 11:40:54.550096 11198 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786016169045816-21563-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786016169045816-21563-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37937
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786016169045816-21563-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37937
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11198
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786016169045816-21563-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:40:54.551348 11198 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:40:54.552470 11198 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:40:54.561730 11203 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:40:54.561698 11206 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:40:54.562074 11198 server_base.cc:1034] running on GCE node
W20260806 11:40:54.561699 11204 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:40:54.563078 11198 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:40:54.564549 11198 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:40:54.565753 11198 hybrid_clock.cc:648] HybridClock initialized: now 1786016454565738 us; error 30 us; skew 500 ppm
Aug 06 11:40:54 dist-test-slave-7c35 krb5kdc[11184](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016454, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:40:54.568941 11198 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 11:40:54.569969 11198 webserver.cc:460] Webserver started at http://127.0.0.1:33629/ using document root <none> and password file <none>
I20260806 11:40:54.570566 11198 fs_manager.cc:359] Metadata directory not provided
I20260806 11:40:54.570636 11198 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:40:54.570823 11198 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:40:54.573263 11198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786016169045816-21563-0/minicluster-data/master-0/data/instance:
uuid: "d8218d1089fd4eb0a350b031842e533b"
format_stamp: "Formatted at 2026-08-06 11:40:54 on dist-test-slave-7c35"
server_key: "e788fa230c4833439454990e3c5e222c"
server_key_iv: "9845b6c80ef6b6aaf4264f41a8efa037"
server_key_version: "encryptionkey@0"
I20260806 11:40:54.573864 11198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786016169045816-21563-0/minicluster-data/master-0/wal/instance:
uuid: "d8218d1089fd4eb0a350b031842e533b"
format_stamp: "Formatted at 2026-08-06 11:40:54 on dist-test-slave-7c35"
server_key: "e788fa230c4833439454990e3c5e222c"
server_key_iv: "9845b6c80ef6b6aaf4264f41a8efa037"
server_key_version: "encryptionkey@0"
I20260806 11:40:54.578047 11198 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260806 11:40:54.581122 11213 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:54.582511 11198 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 11:40:54.582653 11198 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786016169045816-21563-0/minicluster-data/master-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786016169045816-21563-0/minicluster-data/master-0/wal
uuid: "d8218d1089fd4eb0a350b031842e533b"
format_stamp: "Formatted at 2026-08-06 11:40:54 on dist-test-slave-7c35"
server_key: "e788fa230c4833439454990e3c5e222c"
server_key_iv: "9845b6c80ef6b6aaf4264f41a8efa037"
server_key_version: "encryptionkey@0"
I20260806 11:40:54.582784 11198 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786016169045816-21563-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786016169045816-21563-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786016169045816-21563-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:40:54.612644 11198 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:40:54.616426 11198 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:40:54.616670 11198 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:40:54.624678 11198 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37937
I20260806 11:40:54.624706 11268 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37937 every 8 connection(s)
I20260806 11:40:54.625742 11198 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
I20260806 11:40:54.628695 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 11198
I20260806 11:40:54.628717 11269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:40:54.628880 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786016169045816-21563-0/minicluster-data/master-0/wal/instance
I20260806 11:40:54.629180 21563 external_mini_cluster.cc:1428] Setting key cda2d00926621969be7eb32416740806
I20260806 11:40:54.636585 11269 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8218d1089fd4eb0a350b031842e533b: Bootstrap starting.
I20260806 11:40:54.639714 11269 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d8218d1089fd4eb0a350b031842e533b: Neither blocks nor log segments found. Creating new log.
I20260806 11:40:54.640352 11272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:54.630682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58857 (local address 127.0.0.1:37937)
0806 11:40:54.631192 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:54.631212 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:54.631258 (+ 46us) server_negotiation.cc:407] Connection header received
0806 11:40:54.632355 (+ 1097us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:54.632382 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:54.632849 (+ 467us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:54.633255 (+ 406us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:40:54.633718 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:54.635291 (+ 1573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:54.636288 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:54.636988 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:54.637151 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:54.638098 (+ 947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:40:54.638106 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:40:54.638127 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 11:40:54.638659 (+ 532us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:40:54.638733 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 11:40:54.638765 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:40:54.638906 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:54.639098 (+ 192us) server_negotiation.cc:299] Negotiation successful
0806 11:40:54.639325 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":187,"threads_started":1}
I20260806 11:40:54.640663 11269 log.cc:824] T 00000000000000000000000000000000 P d8218d1089fd4eb0a350b031842e533b: Log is configured to *not* fsync() on all Append() calls
I20260806 11:40:54.643114 11269 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8218d1089fd4eb0a350b031842e533b: No bootstrap required, opened a new log
I20260806 11:40:54.643666 21563 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 11:40:54.648170 11269 raft_consensus.cc:348] T 00000000000000000000000000000000 P d8218d1089fd4eb0a350b031842e533b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8218d1089fd4eb0a350b031842e533b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37937 } }
I20260806 11:40:54.648530 11269 raft_consensus.cc:374] T 00000000000000000000000000000000 P d8218d1089fd4eb0a350b031842e533b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:40:54.648594 11269 raft_consensus.cc:729] T 00000000000000000000000000000000 P d8218d1089fd4eb0a350b031842e533b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8218d1089fd4eb0a350b031842e533b, State: Initialized, Role: FOLLOWER
I20260806 11:40:54.649063 11269 consensus_queue.cc:260] T 00000000000000000000000000000000 P d8218d1089fd4eb0a350b031842e533b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8218d1089fd4eb0a350b031842e533b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37937 } }
I20260806 11:40:54.649430 11269 raft_consensus.cc:388] T 00000000000000000000000000000000 P d8218d1089fd4eb0a350b031842e533b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 11:40:54.649533 11269 raft_consensus.cc:482] T 00000000000000000000000000000000 P d8218d1089fd4eb0a350b031842e533b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 11:40:54.649657 11269 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d8218d1089fd4eb0a350b031842e533b [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:40:54.650709 11272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:54.646010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58455 (local address 127.0.0.1:37937)
0806 11:40:54.646162 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:54.646167 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:54.646430 (+ 263us) server_negotiation.cc:407] Connection header received
0806 11:40:54.646591 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:54.646593 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:54.646637 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:54.646705 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:40:54.647052 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:54.647604 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:54.648702 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:54.649021 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:54.649107 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:54.650250 (+ 1143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:40:54.650253 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:40:54.650255 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 11:40:54.650267 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:40:54.650294 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 11:40:54.650304 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:40:54.650366 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:54.650490 (+ 124us) server_negotiation.cc:299] Negotiation successful
0806 11:40:54.650555 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 11:40:54.651170 11269 raft_consensus.cc:504] T 00000000000000000000000000000000 P d8218d1089fd4eb0a350b031842e533b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8218d1089fd4eb0a350b031842e533b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37937 } }
I20260806 11:40:54.651551 11269 leader_election.cc:302] T 00000000000000000000000000000000 P d8218d1089fd4eb0a350b031842e533b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d8218d1089fd4eb0a350b031842e533b; no voters:
I20260806 11:40:54.651913 11269 leader_election.cc:288] T 00000000000000000000000000000000 P d8218d1089fd4eb0a350b031842e533b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 11:40:54.652035 11279 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d8218d1089fd4eb0a350b031842e533b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:40:54.652220 11279 raft_consensus.cc:686] T 00000000000000000000000000000000 P d8218d1089fd4eb0a350b031842e533b [term 1 LEADER]: Becoming Leader. State: Replica: d8218d1089fd4eb0a350b031842e533b, State: Running, Role: LEADER
I20260806 11:40:54.652551 11279 consensus_queue.cc:237] T 00000000000000000000000000000000 P d8218d1089fd4eb0a350b031842e533b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8218d1089fd4eb0a350b031842e533b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37937 } }
I20260806 11:40:54.653368 11269 sys_catalog.cc:564] T 00000000000000000000000000000000 P d8218d1089fd4eb0a350b031842e533b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 11:40:54.654383 11281 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8218d1089fd4eb0a350b031842e533b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d8218d1089fd4eb0a350b031842e533b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8218d1089fd4eb0a350b031842e533b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37937 } } }
I20260806 11:40:54.654644 11282 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8218d1089fd4eb0a350b031842e533b [sys.catalog]: SysCatalogTable state changed. Reason: New leader d8218d1089fd4eb0a350b031842e533b. Latest consensus state: current_term: 1 leader_uuid: "d8218d1089fd4eb0a350b031842e533b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8218d1089fd4eb0a350b031842e533b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37937 } } }
I20260806 11:40:54.654806 11282 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8218d1089fd4eb0a350b031842e533b [sys.catalog]: This master's current role is: LEADER
I20260806 11:40:54.654767 11281 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8218d1089fd4eb0a350b031842e533b [sys.catalog]: This master's current role is: LEADER
I20260806 11:40:54.655983 11289 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 11:40:54.659332 11289 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 11:40:54.664700 11289 catalog_manager.cc:1347] Generated new cluster ID: 47d116f5504648b9ac5b6814fd91c585
I20260806 11:40:54.664775 11289 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 11:40:54.682078 11289 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 11:40:54.683492 11289 catalog_manager.cc:1504] Loading token signing keys...
I20260806 11:40:54.692799 11289 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d8218d1089fd4eb0a350b031842e533b: Generated new TSK 0
I20260806 11:40:54.693765 11289 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 11:40:54.704505 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 11198
2026-08-06T11:40:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1525 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786016169045816-21563-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 11:40:54 dist-test-slave-7c35 krb5kdc[11301](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 11:40:54 dist-test-slave-7c35 krb5kdc[11301](info): set up 2 sockets
Aug 06 11:40:54 dist-test-slave-7c35 krb5kdc[11301](info): commencing operation
krb5kdc: starting...
W20260806 11:40:55.826205 21563 mini_kdc.cc:121] Time spent starting KDC: real 1.098s user 0.005s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Aug 06 11:40:55 dist-test-slave-7c35 krb5kdc[11301](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016455, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T11:40:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T11:40:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786016169045816-21563-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786016169045816-21563-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 11:40:55.971431 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786016169045816-21563-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786016169045816-21563-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786016169045816-21563-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44013
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786016169045816-21563-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44013
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:40:56.095063 11315 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:40:56.095435 11315 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:40:56.095523 11315 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:40:56.101063 11315 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 11:40:56.101207 11315 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:40:56.101250 11315 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:40:56.101294 11315 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 11:40:56.101331 11315 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 11:40:56.108059 11315 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786016169045816-21563-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786016169045816-21563-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44013
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786016169045816-21563-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44013
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11315
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786016169045816-21563-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:40:56.109362 11315 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:40:56.110553 11315 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:40:56.121006 11321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:40:56.120980 11322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:40:56.121109 11315 server_base.cc:1034] running on GCE node
W20260806 11:40:56.120980 11325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:40:56.121994 11315 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:40:56.123490 11315 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:40:56.124703 11315 hybrid_clock.cc:648] HybridClock initialized: now 1786016456124683 us; error 36 us; skew 500 ppm
Aug 06 11:40:56 dist-test-slave-7c35 krb5kdc[11301](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016456, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:40:56.128461 11315 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 11:40:56.129709 11315 webserver.cc:460] Webserver started at http://127.0.0.1:36367/ using document root <none> and password file <none>
I20260806 11:40:56.130457 11315 fs_manager.cc:359] Metadata directory not provided
I20260806 11:40:56.130535 11315 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:40:56.130779 11315 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:40:56.133553 11315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786016169045816-21563-0/minicluster-data/master-0/data/instance:
uuid: "668e75836edf425dac5614dcfe8309c0"
format_stamp: "Formatted at 2026-08-06 11:40:56 on dist-test-slave-7c35"
server_key: "08177b4380000dd49f4494b53fd3732e"
server_key_iv: "a1cb6d4a2d46bf9696e70e8919a32690"
server_key_version: "encryptionkey@0"
I20260806 11:40:56.134305 11315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786016169045816-21563-0/minicluster-data/master-0/wal/instance:
uuid: "668e75836edf425dac5614dcfe8309c0"
format_stamp: "Formatted at 2026-08-06 11:40:56 on dist-test-slave-7c35"
server_key: "08177b4380000dd49f4494b53fd3732e"
server_key_iv: "a1cb6d4a2d46bf9696e70e8919a32690"
server_key_version: "encryptionkey@0"
I20260806 11:40:56.138917 11315 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.002s
I20260806 11:40:56.141927 11332 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:56.143565 11315 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 11:40:56.143728 11315 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786016169045816-21563-0/minicluster-data/master-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786016169045816-21563-0/minicluster-data/master-0/wal
uuid: "668e75836edf425dac5614dcfe8309c0"
format_stamp: "Formatted at 2026-08-06 11:40:56 on dist-test-slave-7c35"
server_key: "08177b4380000dd49f4494b53fd3732e"
server_key_iv: "a1cb6d4a2d46bf9696e70e8919a32690"
server_key_version: "encryptionkey@0"
I20260806 11:40:56.143869 11315 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786016169045816-21563-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786016169045816-21563-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786016169045816-21563-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:40:56.149206 11315 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:40:56.153151 11315 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:40:56.153421 11315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:40:56.161705 11315 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44013
I20260806 11:40:56.161715 11385 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44013 every 8 connection(s)
I20260806 11:40:56.162743 11315 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
I20260806 11:40:56.165891 11386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:40:56.170284 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 11315
I20260806 11:40:56.170452 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786016169045816-21563-0/minicluster-data/master-0/wal/instance
I20260806 11:40:56.170828 21563 external_mini_cluster.cc:1428] Setting key 223d5169aa2a27feb56ebe9f15f95904
I20260806 11:40:56.173223 11386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 668e75836edf425dac5614dcfe8309c0: Bootstrap starting.
I20260806 11:40:56.176618 11386 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 668e75836edf425dac5614dcfe8309c0: Neither blocks nor log segments found. Creating new log.
I20260806 11:40:56.177282 11389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:56.172717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42485 (local address 127.0.0.1:44013)
0806 11:40:56.173256 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:56.173272 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:56.173334 (+ 62us) server_negotiation.cc:407] Connection header received
0806 11:40:56.174069 (+ 735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:56.174090 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:56.174413 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:56.174732 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:40:56.175743 (+ 1011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:40:56.175748 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:40:56.175762 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 11:40:56.175773 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:40:56.175832 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 11:40:56.175854 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:40:56.175949 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:56.176142 (+ 193us) server_negotiation.cc:299] Negotiation successful
0806 11:40:56.176376 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":211,"threads_started":1}
I20260806 11:40:56.177536 11386 log.cc:824] T 00000000000000000000000000000000 P 668e75836edf425dac5614dcfe8309c0: Log is configured to *not* fsync() on all Append() calls
I20260806 11:40:56.180029 11386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 668e75836edf425dac5614dcfe8309c0: No bootstrap required, opened a new log
I20260806 11:40:56.180486 21563 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 11:40:56.184484 11389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:56.182723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60119 (local address 127.0.0.1:44013)
0806 11:40:56.182894 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:56.182899 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:56.183045 (+ 146us) server_negotiation.cc:407] Connection header received
0806 11:40:56.183168 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:56.183173 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:56.183224 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:56.183330 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:40:56.184071 (+ 741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:40:56.184075 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:40:56.184077 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 11:40:56.184090 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:40:56.184122 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 11:40:56.184134 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:40:56.184227 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:56.184299 (+ 72us) server_negotiation.cc:299] Negotiation successful
0806 11:40:56.184340 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 11:40:56.184441 11386 raft_consensus.cc:348] T 00000000000000000000000000000000 P 668e75836edf425dac5614dcfe8309c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "668e75836edf425dac5614dcfe8309c0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44013 } }
I20260806 11:40:56.184939 11386 raft_consensus.cc:374] T 00000000000000000000000000000000 P 668e75836edf425dac5614dcfe8309c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:40:56.185047 11386 raft_consensus.cc:729] T 00000000000000000000000000000000 P 668e75836edf425dac5614dcfe8309c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 668e75836edf425dac5614dcfe8309c0, State: Initialized, Role: FOLLOWER
I20260806 11:40:56.185611 11386 consensus_queue.cc:260] T 00000000000000000000000000000000 P 668e75836edf425dac5614dcfe8309c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "668e75836edf425dac5614dcfe8309c0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44013 } }
I20260806 11:40:56.185807 11386 raft_consensus.cc:388] T 00000000000000000000000000000000 P 668e75836edf425dac5614dcfe8309c0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 11:40:56.185880 11386 raft_consensus.cc:482] T 00000000000000000000000000000000 P 668e75836edf425dac5614dcfe8309c0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 11:40:56.185986 11386 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 668e75836edf425dac5614dcfe8309c0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:40:56.187211 11386 raft_consensus.cc:504] T 00000000000000000000000000000000 P 668e75836edf425dac5614dcfe8309c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "668e75836edf425dac5614dcfe8309c0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44013 } }
I20260806 11:40:56.187634 11386 leader_election.cc:302] T 00000000000000000000000000000000 P 668e75836edf425dac5614dcfe8309c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 668e75836edf425dac5614dcfe8309c0; no voters:
I20260806 11:40:56.188030 11386 leader_election.cc:288] T 00000000000000000000000000000000 P 668e75836edf425dac5614dcfe8309c0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 11:40:56.188241 11396 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 668e75836edf425dac5614dcfe8309c0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:40:56.188573 11396 raft_consensus.cc:686] T 00000000000000000000000000000000 P 668e75836edf425dac5614dcfe8309c0 [term 1 LEADER]: Becoming Leader. State: Replica: 668e75836edf425dac5614dcfe8309c0, State: Running, Role: LEADER
I20260806 11:40:56.188949 11396 consensus_queue.cc:237] T 00000000000000000000000000000000 P 668e75836edf425dac5614dcfe8309c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "668e75836edf425dac5614dcfe8309c0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44013 } }
I20260806 11:40:56.189469 11386 sys_catalog.cc:564] T 00000000000000000000000000000000 P 668e75836edf425dac5614dcfe8309c0 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 11:40:56.191397 11398 sys_catalog.cc:455] T 00000000000000000000000000000000 P 668e75836edf425dac5614dcfe8309c0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 668e75836edf425dac5614dcfe8309c0. Latest consensus state: current_term: 1 leader_uuid: "668e75836edf425dac5614dcfe8309c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "668e75836edf425dac5614dcfe8309c0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44013 } } }
I20260806 11:40:56.191589 11398 sys_catalog.cc:458] T 00000000000000000000000000000000 P 668e75836edf425dac5614dcfe8309c0 [sys.catalog]: This master's current role is: LEADER
I20260806 11:40:56.192062 11397 sys_catalog.cc:455] T 00000000000000000000000000000000 P 668e75836edf425dac5614dcfe8309c0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "668e75836edf425dac5614dcfe8309c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "668e75836edf425dac5614dcfe8309c0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44013 } } }
I20260806 11:40:56.192106 11411 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 11:40:56.192423 11397 sys_catalog.cc:458] T 00000000000000000000000000000000 P 668e75836edf425dac5614dcfe8309c0 [sys.catalog]: This master's current role is: LEADER
I20260806 11:40:56.196043 11411 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 11:40:56.202638 11411 catalog_manager.cc:1347] Generated new cluster ID: 6fcbae7fad71489aaa814e302c2180b4
I20260806 11:40:56.202747 11411 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 11:40:56.221170 11411 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 11:40:56.221565 11411 catalog_manager.cc:1504] Loading token signing keys...
I20260806 11:40:56.229573 11411 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 668e75836edf425dac5614dcfe8309c0: Generated new TSK 0
I20260806 11:40:56.230499 11411 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 11:40:56.246315 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 11315
2026-08-06T11:40:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1540 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786016169045816-21563-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 11:40:56 dist-test-slave-7c35 krb5kdc[11418](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 11:40:56 dist-test-slave-7c35 krb5kdc[11418](info): set up 2 sockets
Aug 06 11:40:56 dist-test-slave-7c35 krb5kdc[11418](info): commencing operation
krb5kdc: starting...
W20260806 11:40:57.338330 21563 mini_kdc.cc:121] Time spent starting KDC: real 1.068s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Aug 06 11:40:57 dist-test-slave-7c35 krb5kdc[11418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016457, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T11:40:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T11:40:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786016169045816-21563-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786016169045816-21563-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 11:40:57.482568 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786016169045816-21563-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786016169045816-21563-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786016169045816-21563-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33465
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786016169045816-21563-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33465
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:40:57.610342 11432 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:40:57.610762 11432 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:40:57.610916 11432 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:40:57.618022 11432 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 11:40:57.618178 11432 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:40:57.618234 11432 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:40:57.618291 11432 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 11:40:57.618350 11432 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 11:40:57.626531 11432 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786016169045816-21563-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786016169045816-21563-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33465
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786016169045816-21563-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33465
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11432
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786016169045816-21563-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:40:57.627888 11432 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:40:57.629048 11432 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 11:40:57.639035 11432 server_base.cc:1034] running on GCE node
W20260806 11:40:57.638993 11441 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:40:57.639048 11439 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:40:57.638993 11437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:40:57.640113 11432 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:40:57.641402 11432 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:40:57.642609 11432 hybrid_clock.cc:648] HybridClock initialized: now 1786016457642577 us; error 47 us; skew 500 ppm
Aug 06 11:40:57 dist-test-slave-7c35 krb5kdc[11418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016457, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:40:57.652173 11432 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 11:40:57.653342 11432 webserver.cc:460] Webserver started at http://127.0.0.1:40041/ using document root <none> and password file <none>
I20260806 11:40:57.654029 11432 fs_manager.cc:359] Metadata directory not provided
I20260806 11:40:57.654103 11432 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:40:57.654345 11432 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:40:57.656883 11432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786016169045816-21563-0/minicluster-data/master-0/data/instance:
uuid: "7f2bd063d8df4b12a4fde0938c971fa6"
format_stamp: "Formatted at 2026-08-06 11:40:57 on dist-test-slave-7c35"
server_key: "28b4fdf30d1de9fa46d0d5bce44f0f72"
server_key_iv: "47f67865817af961cd472867c4162f40"
server_key_version: "encryptionkey@0"
I20260806 11:40:57.657536 11432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786016169045816-21563-0/minicluster-data/master-0/wal/instance:
uuid: "7f2bd063d8df4b12a4fde0938c971fa6"
format_stamp: "Formatted at 2026-08-06 11:40:57 on dist-test-slave-7c35"
server_key: "28b4fdf30d1de9fa46d0d5bce44f0f72"
server_key_iv: "47f67865817af961cd472867c4162f40"
server_key_version: "encryptionkey@0"
I20260806 11:40:57.661993 11432 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260806 11:40:57.665154 11448 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:57.666706 11432 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 11:40:57.666896 11432 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786016169045816-21563-0/minicluster-data/master-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786016169045816-21563-0/minicluster-data/master-0/wal
uuid: "7f2bd063d8df4b12a4fde0938c971fa6"
format_stamp: "Formatted at 2026-08-06 11:40:57 on dist-test-slave-7c35"
server_key: "28b4fdf30d1de9fa46d0d5bce44f0f72"
server_key_iv: "47f67865817af961cd472867c4162f40"
server_key_version: "encryptionkey@0"
I20260806 11:40:57.667049 11432 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786016169045816-21563-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786016169045816-21563-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786016169045816-21563-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:40:57.716287 11432 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:40:57.719779 11432 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:40:57.720008 11432 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:40:57.727974 11432 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33465
I20260806 11:40:57.728001 11502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33465 every 8 connection(s)
I20260806 11:40:57.729424 11432 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
I20260806 11:40:57.732316 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 11432
I20260806 11:40:57.732496 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786016169045816-21563-0/minicluster-data/master-0/wal/instance
I20260806 11:40:57.732839 21563 external_mini_cluster.cc:1428] Setting key 029ed7d92737c3d06cfaff96ce652558
I20260806 11:40:57.733476 11503 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:40:57.742604 11503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f2bd063d8df4b12a4fde0938c971fa6: Bootstrap starting.
Aug 06 11:40:57 dist-test-slave-7c35 krb5kdc[11418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016457, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 11:40:57.745959 11503 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7f2bd063d8df4b12a4fde0938c971fa6: Neither blocks nor log segments found. Creating new log.
I20260806 11:40:57.746887 11503 log.cc:824] T 00000000000000000000000000000000 P 7f2bd063d8df4b12a4fde0938c971fa6: Log is configured to *not* fsync() on all Append() calls
I20260806 11:40:57.749657 11503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f2bd063d8df4b12a4fde0938c971fa6: No bootstrap required, opened a new log
I20260806 11:40:57.752135 11506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:57.734698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.41:35733 (local address 127.0.0.1:33465)
0806 11:40:57.735338 (+ 640us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:57.735360 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:57.735407 (+ 47us) server_negotiation.cc:407] Connection header received
0806 11:40:57.736310 (+ 903us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:57.736329 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:57.736831 (+ 502us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:57.737270 (+ 439us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:40:57.738137 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:57.739876 (+ 1739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:57.741132 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:57.741987 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:57.742178 (+ 191us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:57.745333 (+ 3155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:40:57.745339 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:40:57.745354 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:40:57.745413 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:40:57.748007 (+ 2594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:40:57.748804 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:40:57.748814 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:40:57.748821 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:40:57.749003 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:40:57.749485 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:40:57.749489 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:40:57.749491 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:40:57.750062 (+ 571us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:40:57.750295 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:57.750735 (+ 440us) server_negotiation.cc:299] Negotiation successful
0806 11:40:57.751054 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":439,"thread_start_us":193,"threads_started":1}
I20260806 11:40:57.755048 11503 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7f2bd063d8df4b12a4fde0938c971fa6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f2bd063d8df4b12a4fde0938c971fa6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33465 } }
I20260806 11:40:57.755401 11503 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7f2bd063d8df4b12a4fde0938c971fa6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:40:57.755475 11503 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7f2bd063d8df4b12a4fde0938c971fa6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f2bd063d8df4b12a4fde0938c971fa6, State: Initialized, Role: FOLLOWER
I20260806 11:40:57.755766 21563 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 11:40:57.756048 11503 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7f2bd063d8df4b12a4fde0938c971fa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f2bd063d8df4b12a4fde0938c971fa6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33465 } }
I20260806 11:40:57.756196 11503 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7f2bd063d8df4b12a4fde0938c971fa6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 11:40:57.756253 11503 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7f2bd063d8df4b12a4fde0938c971fa6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 11:40:57.756353 11503 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7f2bd063d8df4b12a4fde0938c971fa6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:40:57.757764 11503 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7f2bd063d8df4b12a4fde0938c971fa6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f2bd063d8df4b12a4fde0938c971fa6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33465 } }
I20260806 11:40:57.758172 11503 leader_election.cc:302] T 00000000000000000000000000000000 P 7f2bd063d8df4b12a4fde0938c971fa6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7f2bd063d8df4b12a4fde0938c971fa6; no voters:
I20260806 11:40:57.758538 11503 leader_election.cc:288] T 00000000000000000000000000000000 P 7f2bd063d8df4b12a4fde0938c971fa6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 11:40:57.758651 11513 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7f2bd063d8df4b12a4fde0938c971fa6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:40:57.758891 11513 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7f2bd063d8df4b12a4fde0938c971fa6 [term 1 LEADER]: Becoming Leader. State: Replica: 7f2bd063d8df4b12a4fde0938c971fa6, State: Running, Role: LEADER
I20260806 11:40:57.759235 11513 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7f2bd063d8df4b12a4fde0938c971fa6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f2bd063d8df4b12a4fde0938c971fa6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33465 } }
I20260806 11:40:57.760358 11503 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7f2bd063d8df4b12a4fde0938c971fa6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 11:40:57.760633 11513 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f2bd063d8df4b12a4fde0938c971fa6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7f2bd063d8df4b12a4fde0938c971fa6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f2bd063d8df4b12a4fde0938c971fa6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33465 } } }
I20260806 11:40:57.760752 11513 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f2bd063d8df4b12a4fde0938c971fa6 [sys.catalog]: This master's current role is: LEADER
I20260806 11:40:57.760944 11513 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f2bd063d8df4b12a4fde0938c971fa6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7f2bd063d8df4b12a4fde0938c971fa6. Latest consensus state: current_term: 1 leader_uuid: "7f2bd063d8df4b12a4fde0938c971fa6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f2bd063d8df4b12a4fde0938c971fa6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33465 } } }
I20260806 11:40:57.761020 11513 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f2bd063d8df4b12a4fde0938c971fa6 [sys.catalog]: This master's current role is: LEADER
I20260806 11:40:57.762600 11517 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 11:40:57.765503 11517 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 11:40:57.772761 11517 catalog_manager.cc:1347] Generated new cluster ID: f604340ed439473b9bdd6b880497ec99
I20260806 11:40:57.772923 11517 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 11:40:57.773128 11506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:57.758153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.41:47069 (local address 127.0.0.1:33465)
0806 11:40:57.758302 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:57.758307 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:57.758626 (+ 319us) server_negotiation.cc:407] Connection header received
0806 11:40:57.758893 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:57.758897 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:57.759020 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:57.759130 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:40:57.760154 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:57.761180 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:57.762190 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:57.762548 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:57.762705 (+ 157us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:57.768354 (+ 5649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:40:57.768358 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:40:57.768361 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:40:57.768396 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:40:57.770259 (+ 1863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:40:57.770775 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:40:57.770778 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:40:57.770780 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:40:57.770926 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:40:57.771347 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:40:57.771351 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:40:57.771355 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:40:57.771589 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:40:57.771732 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:57.772596 (+ 864us) server_negotiation.cc:299] Negotiation successful
0806 11:40:57.772712 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 11:40:57.801198 11517 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 11:40:57.802527 11517 catalog_manager.cc:1504] Loading token signing keys...
I20260806 11:40:57.809351 11517 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7f2bd063d8df4b12a4fde0938c971fa6: Generated new TSK 0
I20260806 11:40:57.810115 11517 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 11:40:57.816612 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 11432
W20260806 11:40:57.824625 11416 connection.cc:535] client connection to 127.0.0.1:33465 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33465): Connection reset by peer (error 104)
2026-08-06T11:40:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1570 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786016169045816-21563-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 11:40:57 dist-test-slave-7c35 krb5kdc[11534](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 11:40:57 dist-test-slave-7c35 krb5kdc[11534](info): set up 2 sockets
Aug 06 11:40:57 dist-test-slave-7c35 krb5kdc[11534](info): commencing operation
krb5kdc: starting...
W20260806 11:40:58.922076 21563 mini_kdc.cc:121] Time spent starting KDC: real 1.082s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Aug 06 11:40:59 dist-test-slave-7c35 krb5kdc[11534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016459, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T11:40:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T11:40:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786016169045816-21563-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786016169045816-21563-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 11:40:59.063350 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786016169045816-21563-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786016169045816-21563-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786016169045816-21563-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33865
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786016169045816-21563-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33865
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:40:59.181162 11548 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:40:59.181455 11548 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:40:59.181500 11548 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:40:59.185788 11548 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 11:40:59.185866 11548 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:40:59.185887 11548 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:40:59.185905 11548 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 11:40:59.185921 11548 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 11:40:59.191730 11548 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786016169045816-21563-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786016169045816-21563-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33865
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786016169045816-21563-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33865
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11548
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786016169045816-21563-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:40:59.192937 11548 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:40:59.194038 11548 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:40:59.203303 11553 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:40:59.203303 11554 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:40:59.203318 11556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:40:59.203577 11548 server_base.cc:1034] running on GCE node
I20260806 11:40:59.204303 11548 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:40:59.205482 11548 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:40:59.206682 11548 hybrid_clock.cc:648] HybridClock initialized: now 1786016459206653 us; error 54 us; skew 500 ppm
Aug 06 11:40:59 dist-test-slave-7c35 krb5kdc[11534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016459, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:40:59.209829 11548 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 11:40:59.211047 11548 webserver.cc:460] Webserver started at http://127.0.0.1:41799/ using document root <none> and password file <none>
I20260806 11:40:59.211666 11548 fs_manager.cc:359] Metadata directory not provided
I20260806 11:40:59.211760 11548 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:40:59.212020 11548 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:40:59.214277 11548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786016169045816-21563-0/minicluster-data/master-0/data/instance:
uuid: "056a3cbd12a9486d86cf597512659fbf"
format_stamp: "Formatted at 2026-08-06 11:40:59 on dist-test-slave-7c35"
server_key: "e2266218fa64f7840753b22f3bf85a85"
server_key_iv: "895b73a14d22490b459dbfb62a6bea1d"
server_key_version: "encryptionkey@0"
I20260806 11:40:59.215029 11548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786016169045816-21563-0/minicluster-data/master-0/wal/instance:
uuid: "056a3cbd12a9486d86cf597512659fbf"
format_stamp: "Formatted at 2026-08-06 11:40:59 on dist-test-slave-7c35"
server_key: "e2266218fa64f7840753b22f3bf85a85"
server_key_iv: "895b73a14d22490b459dbfb62a6bea1d"
server_key_version: "encryptionkey@0"
I20260806 11:40:59.219220 11548 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 11:40:59.222036 11563 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:40:59.223469 11548 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 11:40:59.223621 11548 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786016169045816-21563-0/minicluster-data/master-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786016169045816-21563-0/minicluster-data/master-0/wal
uuid: "056a3cbd12a9486d86cf597512659fbf"
format_stamp: "Formatted at 2026-08-06 11:40:59 on dist-test-slave-7c35"
server_key: "e2266218fa64f7840753b22f3bf85a85"
server_key_iv: "895b73a14d22490b459dbfb62a6bea1d"
server_key_version: "encryptionkey@0"
I20260806 11:40:59.223743 11548 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786016169045816-21563-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786016169045816-21563-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786016169045816-21563-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:40:59.237684 11548 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:40:59.240983 11548 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:40:59.241272 11548 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:40:59.249351 11548 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33865
I20260806 11:40:59.249410 11618 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33865 every 8 connection(s)
I20260806 11:40:59.250497 11548 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
I20260806 11:40:59.251582 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 11548
I20260806 11:40:59.251744 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786016169045816-21563-0/minicluster-data/master-0/wal/instance
I20260806 11:40:59.252063 21563 external_mini_cluster.cc:1428] Setting key c80c4832d04eddae2d79980511d270af
I20260806 11:40:59.254273 11619 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:40:59.261951 11619 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 056a3cbd12a9486d86cf597512659fbf: Bootstrap starting.
I20260806 11:40:59.262964 11622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:59.253571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.41:58027 (local address 127.0.0.1:33865)
0806 11:40:59.254025 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:59.254053 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:59.254099 (+ 46us) server_negotiation.cc:407] Connection header received
0806 11:40:59.255023 (+ 924us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:59.255049 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:59.255495 (+ 446us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:59.255877 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:40:59.256301 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:59.257877 (+ 1576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:59.258912 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:59.259639 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:59.259818 (+ 179us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:59.260820 (+ 1002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:40:59.260828 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:40:59.260845 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 11:40:59.261363 (+ 518us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:40:59.261438 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 11:40:59.261468 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:40:59.261615 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:59.261830 (+ 215us) server_negotiation.cc:299] Negotiation successful
0806 11:40:59.261978 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":119,"threads_started":1}
I20260806 11:40:59.264770 11619 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 056a3cbd12a9486d86cf597512659fbf: Neither blocks nor log segments found. Creating new log.
I20260806 11:40:59.265695 11619 log.cc:824] T 00000000000000000000000000000000 P 056a3cbd12a9486d86cf597512659fbf: Log is configured to *not* fsync() on all Append() calls
I20260806 11:40:59.266294 21563 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 11:40:59.268340 11619 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 056a3cbd12a9486d86cf597512659fbf: No bootstrap required, opened a new log
I20260806 11:40:59.273178 11622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:40:59.268676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.41:35107 (local address 127.0.0.1:33865)
0806 11:40:59.268822 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 11:40:59.268827 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:40:59.269085 (+ 258us) server_negotiation.cc:407] Connection header received
0806 11:40:59.269166 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:40:59.269169 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:40:59.269219 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:40:59.269310 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:40:59.269556 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:59.270351 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:59.271309 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:40:59.271649 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:40:59.271753 (+ 104us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:40:59.272638 (+ 885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:40:59.272642 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:40:59.272645 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 11:40:59.272659 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:40:59.272691 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 11:40:59.272703 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:40:59.272799 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 11:40:59.272994 (+ 195us) server_negotiation.cc:299] Negotiation successful
0806 11:40:59.273034 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 11:40:59.273376 11619 raft_consensus.cc:348] T 00000000000000000000000000000000 P 056a3cbd12a9486d86cf597512659fbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "056a3cbd12a9486d86cf597512659fbf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33865 } }
I20260806 11:40:59.273651 11619 raft_consensus.cc:374] T 00000000000000000000000000000000 P 056a3cbd12a9486d86cf597512659fbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:40:59.273715 11619 raft_consensus.cc:729] T 00000000000000000000000000000000 P 056a3cbd12a9486d86cf597512659fbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 056a3cbd12a9486d86cf597512659fbf, State: Initialized, Role: FOLLOWER
I20260806 11:40:59.274199 11619 consensus_queue.cc:260] T 00000000000000000000000000000000 P 056a3cbd12a9486d86cf597512659fbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "056a3cbd12a9486d86cf597512659fbf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33865 } }
I20260806 11:40:59.274326 11619 raft_consensus.cc:388] T 00000000000000000000000000000000 P 056a3cbd12a9486d86cf597512659fbf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 11:40:59.274415 11619 raft_consensus.cc:482] T 00000000000000000000000000000000 P 056a3cbd12a9486d86cf597512659fbf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 11:40:59.274519 11619 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 056a3cbd12a9486d86cf597512659fbf [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:40:59.275607 11619 raft_consensus.cc:504] T 00000000000000000000000000000000 P 056a3cbd12a9486d86cf597512659fbf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "056a3cbd12a9486d86cf597512659fbf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33865 } }
I20260806 11:40:59.275977 11619 leader_election.cc:302] T 00000000000000000000000000000000 P 056a3cbd12a9486d86cf597512659fbf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 056a3cbd12a9486d86cf597512659fbf; no voters:
I20260806 11:40:59.276352 11619 leader_election.cc:288] T 00000000000000000000000000000000 P 056a3cbd12a9486d86cf597512659fbf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 11:40:59.276429 11629 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 056a3cbd12a9486d86cf597512659fbf [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:40:59.276669 11629 raft_consensus.cc:686] T 00000000000000000000000000000000 P 056a3cbd12a9486d86cf597512659fbf [term 1 LEADER]: Becoming Leader. State: Replica: 056a3cbd12a9486d86cf597512659fbf, State: Running, Role: LEADER
I20260806 11:40:59.276996 11629 consensus_queue.cc:237] T 00000000000000000000000000000000 P 056a3cbd12a9486d86cf597512659fbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "056a3cbd12a9486d86cf597512659fbf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33865 } }
I20260806 11:40:59.277779 11619 sys_catalog.cc:564] T 00000000000000000000000000000000 P 056a3cbd12a9486d86cf597512659fbf [sys.catalog]: configured and running, proceeding with master startup.
I20260806 11:40:59.278975 11630 sys_catalog.cc:455] T 00000000000000000000000000000000 P 056a3cbd12a9486d86cf597512659fbf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "056a3cbd12a9486d86cf597512659fbf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "056a3cbd12a9486d86cf597512659fbf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33865 } } }
I20260806 11:40:59.279138 11630 sys_catalog.cc:458] T 00000000000000000000000000000000 P 056a3cbd12a9486d86cf597512659fbf [sys.catalog]: This master's current role is: LEADER
I20260806 11:40:59.279402 11631 sys_catalog.cc:455] T 00000000000000000000000000000000 P 056a3cbd12a9486d86cf597512659fbf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 056a3cbd12a9486d86cf597512659fbf. Latest consensus state: current_term: 1 leader_uuid: "056a3cbd12a9486d86cf597512659fbf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "056a3cbd12a9486d86cf597512659fbf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33865 } } }
I20260806 11:40:59.279510 11631 sys_catalog.cc:458] T 00000000000000000000000000000000 P 056a3cbd12a9486d86cf597512659fbf [sys.catalog]: This master's current role is: LEADER
I20260806 11:40:59.279675 11638 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 11:40:59.284124 11638 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 11:40:59.290644 11638 catalog_manager.cc:1347] Generated new cluster ID: 0db55eac45ad4ba99322082fddb07d95
I20260806 11:40:59.290743 11638 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 11:40:59.306767 11638 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 11:40:59.308161 11638 catalog_manager.cc:1504] Loading token signing keys...
I20260806 11:40:59.315294 11638 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 056a3cbd12a9486d86cf597512659fbf: Generated new TSK 0
I20260806 11:40:59.316293 11638 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 11:40:59.320380 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 11548
W20260806 11:40:59.328924 11532 connection.cc:535] client connection to 127.0.0.1:33865 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33865): Connection reset by peer (error 104)
2026-08-06T11:40:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1504 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786016169045816-21563-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 11:40:59 dist-test-slave-7c35 krb5kdc[11651](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 11:40:59 dist-test-slave-7c35 krb5kdc[11651](info): set up 2 sockets
Aug 06 11:40:59 dist-test-slave-7c35 krb5kdc[11651](info): commencing operation
krb5kdc: starting...
W20260806 11:41:00.409754 21563 mini_kdc.cc:121] Time spent starting KDC: real 1.066s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Aug 06 11:41:00 dist-test-slave-7c35 krb5kdc[11651](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016460, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T11:41:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T11:41:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786016169045816-21563-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786016169045816-21563-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 11:41:00.551347 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786016169045816-21563-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786016169045816-21563-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786016169045816-21563-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35757
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786016169045816-21563-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35757
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:41:00.672266 11664 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:41:00.672632 11664 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:41:00.672719 11664 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:41:00.676854 11664 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 11:41:00.676967 11664 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:41:00.677003 11664 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:41:00.677044 11664 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 11:41:00.677079 11664 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 11:41:00.682619 11664 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786016169045816-21563-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786016169045816-21563-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35757
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786016169045816-21563-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35757
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11664
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786016169045816-21563-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:41:00.683890 11664 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:41:00.684998 11664 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:41:00.694478 11671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:41:00.694483 11670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:41:00.694655 11664 server_base.cc:1034] running on GCE node
W20260806 11:41:00.694483 11673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:41:00.695703 11664 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:41:00.697122 11664 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:41:00.698345 11664 hybrid_clock.cc:648] HybridClock initialized: now 1786016460698323 us; error 40 us; skew 500 ppm
Aug 06 11:41:00 dist-test-slave-7c35 krb5kdc[11651](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016460, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:41:00.701669 11664 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 11:41:00.702723 11664 webserver.cc:460] Webserver started at http://127.0.0.1:42375/ using document root <none> and password file <none>
I20260806 11:41:00.703378 11664 fs_manager.cc:359] Metadata directory not provided
I20260806 11:41:00.703449 11664 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:41:00.703645 11664 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:41:00.705842 11664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786016169045816-21563-0/minicluster-data/master-0/data/instance:
uuid: "b6110b3379be48c3a2da415bdf0f8f5b"
format_stamp: "Formatted at 2026-08-06 11:41:00 on dist-test-slave-7c35"
server_key: "4ffa596053e923631d8b36a0fe2795eb"
server_key_iv: "7e3b821509483f73307fb0af57342774"
server_key_version: "encryptionkey@0"
I20260806 11:41:00.706403 11664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786016169045816-21563-0/minicluster-data/master-0/wal/instance:
uuid: "b6110b3379be48c3a2da415bdf0f8f5b"
format_stamp: "Formatted at 2026-08-06 11:41:00 on dist-test-slave-7c35"
server_key: "4ffa596053e923631d8b36a0fe2795eb"
server_key_iv: "7e3b821509483f73307fb0af57342774"
server_key_version: "encryptionkey@0"
I20260806 11:41:00.710431 11664 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 11:41:00.713240 11681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:41:00.714694 11664 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 11:41:00.714815 11664 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786016169045816-21563-0/minicluster-data/master-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786016169045816-21563-0/minicluster-data/master-0/wal
uuid: "b6110b3379be48c3a2da415bdf0f8f5b"
format_stamp: "Formatted at 2026-08-06 11:41:00 on dist-test-slave-7c35"
server_key: "4ffa596053e923631d8b36a0fe2795eb"
server_key_iv: "7e3b821509483f73307fb0af57342774"
server_key_version: "encryptionkey@0"
I20260806 11:41:00.714999 11664 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786016169045816-21563-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786016169045816-21563-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786016169045816-21563-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:41:00.720115 11664 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:41:00.723641 11664 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:41:00.723860 11664 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:41:00.732152 11664 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35757
I20260806 11:41:00.732195 11737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35757 every 8 connection(s)
I20260806 11:41:00.733193 11664 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
I20260806 11:41:00.736248 11738 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:41:00.739748 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 11664
I20260806 11:41:00.739923 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786016169045816-21563-0/minicluster-data/master-0/wal/instance
I20260806 11:41:00.740233 21563 external_mini_cluster.cc:1428] Setting key 65d0734a79c3094937a11c8ad40dbfc1
I20260806 11:41:00.742781 11738 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6110b3379be48c3a2da415bdf0f8f5b: Bootstrap starting.
I20260806 11:41:00.745518 11738 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b6110b3379be48c3a2da415bdf0f8f5b: Neither blocks nor log segments found. Creating new log.
I20260806 11:41:00.745939 11741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:00.741784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.41:59409 (local address 127.0.0.1:35757)
0806 11:41:00.742276 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:00.742291 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:00.742329 (+ 38us) server_negotiation.cc:407] Connection header received
0806 11:41:00.743098 (+ 769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:00.743118 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:00.743439 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:00.743790 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:00.744643 (+ 853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:41:00.744647 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:41:00.744659 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 11:41:00.744670 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:41:00.744726 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 11:41:00.744746 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:41:00.744881 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 11:41:00.744986 (+ 105us) server_negotiation.cc:299] Negotiation successful
0806 11:41:00.745167 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":140,"threads_started":1}
I20260806 11:41:00.746377 11738 log.cc:824] T 00000000000000000000000000000000 P b6110b3379be48c3a2da415bdf0f8f5b: Log is configured to *not* fsync() on all Append() calls
I20260806 11:41:00.748728 11738 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6110b3379be48c3a2da415bdf0f8f5b: No bootstrap required, opened a new log
I20260806 11:41:00.749012 21563 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 11:41:00.753165 11741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:00.751483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.41:56719 (local address 127.0.0.1:35757)
0806 11:41:00.751620 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:00.751625 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:00.751810 (+ 185us) server_negotiation.cc:407] Connection header received
0806 11:41:00.751893 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:00.751897 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:00.751938 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:00.752038 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:00.752757 (+ 719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:41:00.752761 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:41:00.752764 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 11:41:00.752777 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:41:00.752809 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 11:41:00.752822 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:41:00.752920 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 11:41:00.752981 (+ 61us) server_negotiation.cc:299] Negotiation successful
0806 11:41:00.753029 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 11:41:00.753845 11738 raft_consensus.cc:348] T 00000000000000000000000000000000 P b6110b3379be48c3a2da415bdf0f8f5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6110b3379be48c3a2da415bdf0f8f5b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35757 } }
I20260806 11:41:00.754179 11738 raft_consensus.cc:374] T 00000000000000000000000000000000 P b6110b3379be48c3a2da415bdf0f8f5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:41:00.754251 11738 raft_consensus.cc:729] T 00000000000000000000000000000000 P b6110b3379be48c3a2da415bdf0f8f5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6110b3379be48c3a2da415bdf0f8f5b, State: Initialized, Role: FOLLOWER
I20260806 11:41:00.754748 11738 consensus_queue.cc:260] T 00000000000000000000000000000000 P b6110b3379be48c3a2da415bdf0f8f5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6110b3379be48c3a2da415bdf0f8f5b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35757 } }
I20260806 11:41:00.754936 11738 raft_consensus.cc:388] T 00000000000000000000000000000000 P b6110b3379be48c3a2da415bdf0f8f5b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 11:41:00.755035 11738 raft_consensus.cc:482] T 00000000000000000000000000000000 P b6110b3379be48c3a2da415bdf0f8f5b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 11:41:00.755146 11738 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b6110b3379be48c3a2da415bdf0f8f5b [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:41:00.756291 11738 raft_consensus.cc:504] T 00000000000000000000000000000000 P b6110b3379be48c3a2da415bdf0f8f5b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6110b3379be48c3a2da415bdf0f8f5b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35757 } }
I20260806 11:41:00.756680 11738 leader_election.cc:302] T 00000000000000000000000000000000 P b6110b3379be48c3a2da415bdf0f8f5b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b6110b3379be48c3a2da415bdf0f8f5b; no voters:
I20260806 11:41:00.757037 11738 leader_election.cc:288] T 00000000000000000000000000000000 P b6110b3379be48c3a2da415bdf0f8f5b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 11:41:00.757191 11748 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b6110b3379be48c3a2da415bdf0f8f5b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:41:00.757449 11748 raft_consensus.cc:686] T 00000000000000000000000000000000 P b6110b3379be48c3a2da415bdf0f8f5b [term 1 LEADER]: Becoming Leader. State: Replica: b6110b3379be48c3a2da415bdf0f8f5b, State: Running, Role: LEADER
I20260806 11:41:00.757792 11748 consensus_queue.cc:237] T 00000000000000000000000000000000 P b6110b3379be48c3a2da415bdf0f8f5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6110b3379be48c3a2da415bdf0f8f5b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35757 } }
I20260806 11:41:00.758487 11738 sys_catalog.cc:564] T 00000000000000000000000000000000 P b6110b3379be48c3a2da415bdf0f8f5b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 11:41:00.759902 11750 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6110b3379be48c3a2da415bdf0f8f5b [sys.catalog]: SysCatalogTable state changed. Reason: New leader b6110b3379be48c3a2da415bdf0f8f5b. Latest consensus state: current_term: 1 leader_uuid: "b6110b3379be48c3a2da415bdf0f8f5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6110b3379be48c3a2da415bdf0f8f5b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35757 } } }
I20260806 11:41:00.759932 11749 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6110b3379be48c3a2da415bdf0f8f5b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b6110b3379be48c3a2da415bdf0f8f5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6110b3379be48c3a2da415bdf0f8f5b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35757 } } }
I20260806 11:41:00.760099 11749 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6110b3379be48c3a2da415bdf0f8f5b [sys.catalog]: This master's current role is: LEADER
I20260806 11:41:00.760358 11750 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6110b3379be48c3a2da415bdf0f8f5b [sys.catalog]: This master's current role is: LEADER
I20260806 11:41:00.760568 11758 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 11:41:00.764182 11758 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 11:41:00.770044 11758 catalog_manager.cc:1347] Generated new cluster ID: 3fc1918f070d48d49f9786f299628466
I20260806 11:41:00.770174 11758 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 11:41:00.805450 11758 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 11:41:00.805855 11758 catalog_manager.cc:1504] Loading token signing keys...
I20260806 11:41:00.813447 11758 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b6110b3379be48c3a2da415bdf0f8f5b: Generated new TSK 0
I20260806 11:41:00.814461 11758 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 11:41:00.831139 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 11664
2026-08-06T11:41:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1510 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786016169045816-21563-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 11:41:00 dist-test-slave-7c35 krb5kdc[11769](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 11:41:00 dist-test-slave-7c35 krb5kdc[11769](info): set up 2 sockets
Aug 06 11:41:00 dist-test-slave-7c35 krb5kdc[11769](info): commencing operation
krb5kdc: starting...
W20260806 11:41:01.909920 21563 mini_kdc.cc:121] Time spent starting KDC: real 1.056s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Aug 06 11:41:02 dist-test-slave-7c35 krb5kdc[11769](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016462, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T11:41:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T11:41:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786016169045816-21563-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786016169045816-21563-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 11:41:02.060015 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786016169045816-21563-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786016169045816-21563-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786016169045816-21563-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44865
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786016169045816-21563-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44865
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:41:02.185711 11783 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:41:02.186111 11783 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:41:02.186198 11783 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:41:02.190511 11783 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 11:41:02.190640 11783 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:41:02.190685 11783 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:41:02.190733 11783 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 11:41:02.190775 11783 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 11:41:02.196537 11783 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786016169045816-21563-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786016169045816-21563-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44865
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786016169045816-21563-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44865
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11783
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786016169045816-21563-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:41:02.197855 11783 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:41:02.199076 11783 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:41:02.209561 11789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:41:02.209702 11783 server_base.cc:1034] running on GCE node
W20260806 11:41:02.209582 11788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:41:02.209653 11791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:41:02.210650 11783 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:41:02.212010 11783 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:41:02.213243 11783 hybrid_clock.cc:648] HybridClock initialized: now 1786016462213209 us; error 86 us; skew 500 ppm
Aug 06 11:41:02 dist-test-slave-7c35 krb5kdc[11769](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016462, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:41:02.216959 11783 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 11:41:02.217942 11783 webserver.cc:460] Webserver started at http://127.0.0.1:33153/ using document root <none> and password file <none>
I20260806 11:41:02.218573 11783 fs_manager.cc:359] Metadata directory not provided
I20260806 11:41:02.218642 11783 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:41:02.218947 11783 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:41:02.221256 11783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786016169045816-21563-0/minicluster-data/master-0/data/instance:
uuid: "acad96a6a20347cd844ac8ed52bcdb66"
format_stamp: "Formatted at 2026-08-06 11:41:02 on dist-test-slave-7c35"
server_key: "b566dbfaf4a3c69c0d3c6e00d731be7e"
server_key_iv: "807491d782715ba893db82344642c5a8"
server_key_version: "encryptionkey@0"
I20260806 11:41:02.221894 11783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786016169045816-21563-0/minicluster-data/master-0/wal/instance:
uuid: "acad96a6a20347cd844ac8ed52bcdb66"
format_stamp: "Formatted at 2026-08-06 11:41:02 on dist-test-slave-7c35"
server_key: "b566dbfaf4a3c69c0d3c6e00d731be7e"
server_key_iv: "807491d782715ba893db82344642c5a8"
server_key_version: "encryptionkey@0"
I20260806 11:41:02.226243 11783 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 11:41:02.229123 11798 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:41:02.230666 11783 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 11:41:02.230829 11783 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786016169045816-21563-0/minicluster-data/master-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786016169045816-21563-0/minicluster-data/master-0/wal
uuid: "acad96a6a20347cd844ac8ed52bcdb66"
format_stamp: "Formatted at 2026-08-06 11:41:02 on dist-test-slave-7c35"
server_key: "b566dbfaf4a3c69c0d3c6e00d731be7e"
server_key_iv: "807491d782715ba893db82344642c5a8"
server_key_version: "encryptionkey@0"
I20260806 11:41:02.231029 11783 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786016169045816-21563-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786016169045816-21563-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786016169045816-21563-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:41:02.250502 11783 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:41:02.254406 11783 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:41:02.254662 11783 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:41:02.262717 11783 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44865
I20260806 11:41:02.262820 11854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44865 every 8 connection(s)
I20260806 11:41:02.263911 11783 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
I20260806 11:41:02.267050 11855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:41:02.269008 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 11783
I20260806 11:41:02.269191 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786016169045816-21563-0/minicluster-data/master-0/wal/instance
I20260806 11:41:02.269531 21563 external_mini_cluster.cc:1428] Setting key 9f4cf1d0de89ecb62716442afd1b9454
I20260806 11:41:02.273901 11855 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P acad96a6a20347cd844ac8ed52bcdb66: Bootstrap starting.
I20260806 11:41:02.276700 11855 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P acad96a6a20347cd844ac8ed52bcdb66: Neither blocks nor log segments found. Creating new log.
I20260806 11:41:02.277643 11855 log.cc:824] T 00000000000000000000000000000000 P acad96a6a20347cd844ac8ed52bcdb66: Log is configured to *not* fsync() on all Append() calls
I20260806 11:41:02.279884 11855 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P acad96a6a20347cd844ac8ed52bcdb66: No bootstrap required, opened a new log
Aug 06 11:41:02 dist-test-slave-7c35 krb5kdc[11769](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016462, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 11:41:02.284587 11855 raft_consensus.cc:348] T 00000000000000000000000000000000 P acad96a6a20347cd844ac8ed52bcdb66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acad96a6a20347cd844ac8ed52bcdb66" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44865 } }
I20260806 11:41:02.284904 11855 raft_consensus.cc:374] T 00000000000000000000000000000000 P acad96a6a20347cd844ac8ed52bcdb66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:41:02.284974 11855 raft_consensus.cc:729] T 00000000000000000000000000000000 P acad96a6a20347cd844ac8ed52bcdb66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acad96a6a20347cd844ac8ed52bcdb66, State: Initialized, Role: FOLLOWER
I20260806 11:41:02.285470 11855 consensus_queue.cc:260] T 00000000000000000000000000000000 P acad96a6a20347cd844ac8ed52bcdb66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acad96a6a20347cd844ac8ed52bcdb66" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44865 } }
I20260806 11:41:02.285642 11855 raft_consensus.cc:388] T 00000000000000000000000000000000 P acad96a6a20347cd844ac8ed52bcdb66 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 11:41:02.285733 11855 raft_consensus.cc:482] T 00000000000000000000000000000000 P acad96a6a20347cd844ac8ed52bcdb66 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 11:41:02.285861 11855 raft_consensus.cc:3037] T 00000000000000000000000000000000 P acad96a6a20347cd844ac8ed52bcdb66 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:41:02.286052 11858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:02.271084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.41:55641 (local address 127.0.0.1:44865)
0806 11:41:02.271544 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:02.271565 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:02.271609 (+ 44us) server_negotiation.cc:407] Connection header received
0806 11:41:02.272672 (+ 1063us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:02.272697 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:02.273161 (+ 464us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:02.273537 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:02.274459 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:02.276025 (+ 1566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:02.277042 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:02.277759 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:02.277970 (+ 211us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:41:02.280783 (+ 2813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:41:02.280788 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:41:02.280803 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:41:02.280831 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:41:02.282518 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:41:02.283217 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:41:02.283222 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:41:02.283233 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:41:02.283392 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:41:02.283763 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:41:02.283766 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:41:02.283767 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:41:02.284191 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:41:02.284420 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0806 11:41:02.284749 (+ 329us) server_negotiation.cc:299] Negotiation successful
0806 11:41:02.285077 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":121,"threads_started":1}
I20260806 11:41:02.286988 11855 raft_consensus.cc:504] T 00000000000000000000000000000000 P acad96a6a20347cd844ac8ed52bcdb66 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acad96a6a20347cd844ac8ed52bcdb66" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44865 } }
I20260806 11:41:02.287369 11855 leader_election.cc:302] T 00000000000000000000000000000000 P acad96a6a20347cd844ac8ed52bcdb66 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: acad96a6a20347cd844ac8ed52bcdb66; no voters:
I20260806 11:41:02.287734 11855 leader_election.cc:288] T 00000000000000000000000000000000 P acad96a6a20347cd844ac8ed52bcdb66 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 11:41:02.287819 11860 raft_consensus.cc:2781] T 00000000000000000000000000000000 P acad96a6a20347cd844ac8ed52bcdb66 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:41:02.288022 11860 raft_consensus.cc:686] T 00000000000000000000000000000000 P acad96a6a20347cd844ac8ed52bcdb66 [term 1 LEADER]: Becoming Leader. State: Replica: acad96a6a20347cd844ac8ed52bcdb66, State: Running, Role: LEADER
I20260806 11:41:02.288322 11860 consensus_queue.cc:237] T 00000000000000000000000000000000 P acad96a6a20347cd844ac8ed52bcdb66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acad96a6a20347cd844ac8ed52bcdb66" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44865 } }
I20260806 11:41:02.289100 11855 sys_catalog.cc:564] T 00000000000000000000000000000000 P acad96a6a20347cd844ac8ed52bcdb66 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 11:41:02.289618 21563 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 11:41:02.290131 11862 sys_catalog.cc:455] T 00000000000000000000000000000000 P acad96a6a20347cd844ac8ed52bcdb66 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "acad96a6a20347cd844ac8ed52bcdb66" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acad96a6a20347cd844ac8ed52bcdb66" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44865 } } }
I20260806 11:41:02.290511 11862 sys_catalog.cc:458] T 00000000000000000000000000000000 P acad96a6a20347cd844ac8ed52bcdb66 [sys.catalog]: This master's current role is: LEADER
I20260806 11:41:02.290409 11861 sys_catalog.cc:455] T 00000000000000000000000000000000 P acad96a6a20347cd844ac8ed52bcdb66 [sys.catalog]: SysCatalogTable state changed. Reason: New leader acad96a6a20347cd844ac8ed52bcdb66. Latest consensus state: current_term: 1 leader_uuid: "acad96a6a20347cd844ac8ed52bcdb66" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acad96a6a20347cd844ac8ed52bcdb66" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44865 } } }
I20260806 11:41:02.290927 11861 sys_catalog.cc:458] T 00000000000000000000000000000000 P acad96a6a20347cd844ac8ed52bcdb66 [sys.catalog]: This master's current role is: LEADER
W20260806 11:41:02.294019 11878 catalog_manager.cc:1558] T 00000000000000000000000000000000 P acad96a6a20347cd844ac8ed52bcdb66: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 11:41:02.294090 11878 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 11:41:02.294185 11870 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 11:41:02.295215 11870 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 11:41:02.301676 11870 catalog_manager.cc:1347] Generated new cluster ID: 8827ca54f50341a7a2344d38c32aa98f
I20260806 11:41:02.301759 11870 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 11:41:02.307838 11858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:02.294842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.41:59597 (local address 127.0.0.1:44865)
0806 11:41:02.295016 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:02.295021 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:02.299275 (+ 4254us) server_negotiation.cc:407] Connection header received
0806 11:41:02.299427 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:02.299430 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:02.299492 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:02.299588 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:02.300391 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:02.300955 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:02.301886 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:02.302216 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:02.302293 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:41:02.304185 (+ 1892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:41:02.304189 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:41:02.304191 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:41:02.304219 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:41:02.305898 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:41:02.306449 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:41:02.306453 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:41:02.306455 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:41:02.306546 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:41:02.306909 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:41:02.306912 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:41:02.306914 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:41:02.307141 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:41:02.307308 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0806 11:41:02.307567 (+ 259us) server_negotiation.cc:299] Negotiation successful
0806 11:41:02.307669 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 11:41:02.316015 11870 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 11:41:02.317039 11870 catalog_manager.cc:1504] Loading token signing keys...
I20260806 11:41:02.328922 11870 catalog_manager.cc:5868] T 00000000000000000000000000000000 P acad96a6a20347cd844ac8ed52bcdb66: Generated new TSK 0
I20260806 11:41:02.329936 11870 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 11:41:02.339067 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 11783
W20260806 11:41:02.346958 11767 connection.cc:535] client connection to 127.0.0.1:44865 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44865): Connection reset by peer (error 104)
2026-08-06T11:41:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1507 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786016169045816-21563-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 11:41:02 dist-test-slave-7c35 krb5kdc[11887](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 11:41:02 dist-test-slave-7c35 krb5kdc[11887](info): set up 2 sockets
Aug 06 11:41:02 dist-test-slave-7c35 krb5kdc[11887](info): commencing operation
krb5kdc: starting...
W20260806 11:41:03.424162 21563 mini_kdc.cc:121] Time spent starting KDC: real 1.062s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Aug 06 11:41:03 dist-test-slave-7c35 krb5kdc[11887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016463, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T11:41:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T11:41:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786016169045816-21563-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786016169045816-21563-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 11:41:03.566838 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786016169045816-21563-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786016169045816-21563-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786016169045816-21563-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34681
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786016169045816-21563-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34681
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:41:03.694082 11900 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:41:03.694404 11900 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:41:03.694453 11900 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:41:03.698590 11900 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 11:41:03.698674 11900 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:41:03.698697 11900 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:41:03.698717 11900 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 11:41:03.698735 11900 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 11:41:03.704416 11900 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786016169045816-21563-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786016169045816-21563-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34681
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786016169045816-21563-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34681
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11900
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786016169045816-21563-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:41:03.705585 11900 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:41:03.706651 11900 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:41:03.715909 11906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:41:03.715860 11905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:41:03.716099 11900 server_base.cc:1034] running on GCE node
W20260806 11:41:03.715860 11908 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:41:03.716885 11900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:41:03.718080 11900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:41:03.719269 11900 hybrid_clock.cc:648] HybridClock initialized: now 1786016463719254 us; error 27 us; skew 500 ppm
Aug 06 11:41:03 dist-test-slave-7c35 krb5kdc[11887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016463, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:41:03.727685 11900 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 11:41:03.728729 11900 webserver.cc:460] Webserver started at http://127.0.0.1:34657/ using document root <none> and password file <none>
I20260806 11:41:03.729360 11900 fs_manager.cc:359] Metadata directory not provided
I20260806 11:41:03.729458 11900 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:41:03.729718 11900 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:41:03.732025 11900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786016169045816-21563-0/minicluster-data/master-0/data/instance:
uuid: "6e2fd471fba649d1b1330c457d91765b"
format_stamp: "Formatted at 2026-08-06 11:41:03 on dist-test-slave-7c35"
server_key: "6907d41d85280b8676e46fbca5ba56e1"
server_key_iv: "2c8395383e4b1074daf0043251d67fb7"
server_key_version: "encryptionkey@0"
I20260806 11:41:03.732631 11900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786016169045816-21563-0/minicluster-data/master-0/wal/instance:
uuid: "6e2fd471fba649d1b1330c457d91765b"
format_stamp: "Formatted at 2026-08-06 11:41:03 on dist-test-slave-7c35"
server_key: "6907d41d85280b8676e46fbca5ba56e1"
server_key_iv: "2c8395383e4b1074daf0043251d67fb7"
server_key_version: "encryptionkey@0"
I20260806 11:41:03.736706 11900 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260806 11:41:03.739440 11915 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:41:03.740788 11900 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 11:41:03.740944 11900 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786016169045816-21563-0/minicluster-data/master-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786016169045816-21563-0/minicluster-data/master-0/wal
uuid: "6e2fd471fba649d1b1330c457d91765b"
format_stamp: "Formatted at 2026-08-06 11:41:03 on dist-test-slave-7c35"
server_key: "6907d41d85280b8676e46fbca5ba56e1"
server_key_iv: "2c8395383e4b1074daf0043251d67fb7"
server_key_version: "encryptionkey@0"
I20260806 11:41:03.741065 11900 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786016169045816-21563-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786016169045816-21563-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786016169045816-21563-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:41:03.765852 11900 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:41:03.769832 11900 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:41:03.770116 11900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:41:03.778760 11900 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34681
I20260806 11:41:03.778807 11970 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34681 every 8 connection(s)
I20260806 11:41:03.779855 11900 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
I20260806 11:41:03.783072 11971 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:41:03.785765 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 11900
I20260806 11:41:03.785938 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786016169045816-21563-0/minicluster-data/master-0/wal/instance
I20260806 11:41:03.786258 21563 external_mini_cluster.cc:1428] Setting key 432dfe37af0221ac5cce45968f907ccb
I20260806 11:41:03.790305 11971 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e2fd471fba649d1b1330c457d91765b: Bootstrap starting.
I20260806 11:41:03.793192 11971 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e2fd471fba649d1b1330c457d91765b: Neither blocks nor log segments found. Creating new log.
I20260806 11:41:03.794247 11971 log.cc:824] T 00000000000000000000000000000000 P 6e2fd471fba649d1b1330c457d91765b: Log is configured to *not* fsync() on all Append() calls
I20260806 11:41:03.796273 11974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:03.787984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.41:36559 (local address 127.0.0.1:34681)
0806 11:41:03.788441 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:03.788468 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:03.788507 (+ 39us) server_negotiation.cc:407] Connection header received
0806 11:41:03.789202 (+ 695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:03.789220 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:03.789608 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:03.789916 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:03.790290 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:03.791434 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:03.792700 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:03.793201 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:03.793407 (+ 206us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:41:03.794330 (+ 923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:41:03.794335 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:41:03.794376 (+ 41us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 11:41:03.794822 (+ 446us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:41:03.794948 (+ 126us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 11:41:03.794976 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:41:03.795136 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0806 11:41:03.795334 (+ 198us) server_negotiation.cc:299] Negotiation successful
0806 11:41:03.795492 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":148,"threads_started":1}
I20260806 11:41:03.796617 11971 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e2fd471fba649d1b1330c457d91765b: No bootstrap required, opened a new log
I20260806 11:41:03.799697 21563 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 11:41:03.801134 11971 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e2fd471fba649d1b1330c457d91765b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e2fd471fba649d1b1330c457d91765b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34681 } }
I20260806 11:41:03.801420 11971 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6e2fd471fba649d1b1330c457d91765b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:41:03.801481 11971 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e2fd471fba649d1b1330c457d91765b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e2fd471fba649d1b1330c457d91765b, State: Initialized, Role: FOLLOWER
I20260806 11:41:03.802011 11971 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e2fd471fba649d1b1330c457d91765b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e2fd471fba649d1b1330c457d91765b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34681 } }
I20260806 11:41:03.802196 11971 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6e2fd471fba649d1b1330c457d91765b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 11:41:03.802266 11971 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e2fd471fba649d1b1330c457d91765b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 11:41:03.802372 11971 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e2fd471fba649d1b1330c457d91765b [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:41:03.803468 11971 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e2fd471fba649d1b1330c457d91765b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e2fd471fba649d1b1330c457d91765b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34681 } }
I20260806 11:41:03.803833 11971 leader_election.cc:302] T 00000000000000000000000000000000 P 6e2fd471fba649d1b1330c457d91765b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6e2fd471fba649d1b1330c457d91765b; no voters:
I20260806 11:41:03.804201 11971 leader_election.cc:288] T 00000000000000000000000000000000 P 6e2fd471fba649d1b1330c457d91765b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 11:41:03.804299 11981 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6e2fd471fba649d1b1330c457d91765b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:41:03.804479 11981 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6e2fd471fba649d1b1330c457d91765b [term 1 LEADER]: Becoming Leader. State: Replica: 6e2fd471fba649d1b1330c457d91765b, State: Running, Role: LEADER
I20260806 11:41:03.804857 11981 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e2fd471fba649d1b1330c457d91765b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e2fd471fba649d1b1330c457d91765b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34681 } }
I20260806 11:41:03.805639 11971 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e2fd471fba649d1b1330c457d91765b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 11:41:03.806634 11983 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e2fd471fba649d1b1330c457d91765b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e2fd471fba649d1b1330c457d91765b. Latest consensus state: current_term: 1 leader_uuid: "6e2fd471fba649d1b1330c457d91765b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e2fd471fba649d1b1330c457d91765b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34681 } } }
I20260806 11:41:03.806805 11974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:03.801881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.41:33011 (local address 127.0.0.1:34681)
0806 11:41:03.802016 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:03.802023 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:03.802255 (+ 232us) server_negotiation.cc:407] Connection header received
0806 11:41:03.802364 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:03.802367 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:03.802421 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:03.802540 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:03.802821 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:03.803397 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:03.804387 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:03.804762 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:03.804858 (+ 96us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:41:03.805959 (+ 1101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:41:03.805963 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:41:03.805966 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 11:41:03.805980 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:41:03.806014 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 11:41:03.806046 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:41:03.806171 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 11:41:03.806594 (+ 423us) server_negotiation.cc:299] Negotiation successful
0806 11:41:03.806637 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":48}
I20260806 11:41:03.806841 11983 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e2fd471fba649d1b1330c457d91765b [sys.catalog]: This master's current role is: LEADER
I20260806 11:41:03.808032 11982 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e2fd471fba649d1b1330c457d91765b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6e2fd471fba649d1b1330c457d91765b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e2fd471fba649d1b1330c457d91765b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34681 } } }
I20260806 11:41:03.808158 11982 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e2fd471fba649d1b1330c457d91765b [sys.catalog]: This master's current role is: LEADER
I20260806 11:41:03.808472 11993 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 11:41:03.811529 11993 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 11:41:03.817435 11993 catalog_manager.cc:1347] Generated new cluster ID: c30c2a7b7dfd46319f149a16524f7815
I20260806 11:41:03.817520 11993 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 11:41:03.838644 11993 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 11:41:03.840152 11993 catalog_manager.cc:1504] Loading token signing keys...
I20260806 11:41:03.854693 11993 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6e2fd471fba649d1b1330c457d91765b: Generated new TSK 0
I20260806 11:41:03.855692 11993 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 11:41:03.863608 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 11900
W20260806 11:41:03.871492 11885 connection.cc:535] client connection to 127.0.0.1:34681 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34681): Connection reset by peer (error 104)
2026-08-06T11:41:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1524 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786016169045816-21563-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 11:41:03 dist-test-slave-7c35 krb5kdc[12003](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 11:41:03 dist-test-slave-7c35 krb5kdc[12003](info): set up 2 sockets
Aug 06 11:41:03 dist-test-slave-7c35 krb5kdc[12003](info): commencing operation
krb5kdc: starting...
W20260806 11:41:04.946446 21563 mini_kdc.cc:121] Time spent starting KDC: real 1.060s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Aug 06 11:41:05 dist-test-slave-7c35 krb5kdc[12003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016465, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T11:41:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T11:41:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786016169045816-21563-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786016169045816-21563-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 11:41:05.091382 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786016169045816-21563-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786016169045816-21563-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786016169045816-21563-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45329
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786016169045816-21563-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45329
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:41:05.212255 12017 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:41:05.212653 12017 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:41:05.212743 12017 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:41:05.217191 12017 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 11:41:05.217315 12017 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:41:05.217355 12017 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:41:05.217422 12017 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 11:41:05.217479 12017 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 11:41:05.223496 12017 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786016169045816-21563-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786016169045816-21563-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45329
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786016169045816-21563-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45329
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12017
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786016169045816-21563-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:41:05.224817 12017 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:41:05.225994 12017 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:41:05.236665 12022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:41:05.236847 12023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:41:05.236862 12017 server_base.cc:1034] running on GCE node
W20260806 11:41:05.236665 12026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:41:05.237782 12017 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:41:05.239171 12017 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:41:05.240402 12017 hybrid_clock.cc:648] HybridClock initialized: now 1786016465240371 us; error 44 us; skew 500 ppm
Aug 06 11:41:05 dist-test-slave-7c35 krb5kdc[12003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016465, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:41:05.244084 12017 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 11:41:05.245139 12017 webserver.cc:460] Webserver started at http://127.0.0.1:35891/ using document root <none> and password file <none>
I20260806 11:41:05.245801 12017 fs_manager.cc:359] Metadata directory not provided
I20260806 11:41:05.245898 12017 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:41:05.246142 12017 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:41:05.248430 12017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786016169045816-21563-0/minicluster-data/master-0/data/instance:
uuid: "709d1c47063348c3bd79bea0732d7add"
format_stamp: "Formatted at 2026-08-06 11:41:05 on dist-test-slave-7c35"
server_key: "657d5584a6d35bb2e2c706a469e4f645"
server_key_iv: "2c6a6f54ddc22da6cc2cbcaf850ea345"
server_key_version: "encryptionkey@0"
I20260806 11:41:05.249044 12017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786016169045816-21563-0/minicluster-data/master-0/wal/instance:
uuid: "709d1c47063348c3bd79bea0732d7add"
format_stamp: "Formatted at 2026-08-06 11:41:05 on dist-test-slave-7c35"
server_key: "657d5584a6d35bb2e2c706a469e4f645"
server_key_iv: "2c6a6f54ddc22da6cc2cbcaf850ea345"
server_key_version: "encryptionkey@0"
I20260806 11:41:05.253301 12017 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 11:41:05.256201 12033 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:41:05.257673 12017 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 11:41:05.257858 12017 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786016169045816-21563-0/minicluster-data/master-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786016169045816-21563-0/minicluster-data/master-0/wal
uuid: "709d1c47063348c3bd79bea0732d7add"
format_stamp: "Formatted at 2026-08-06 11:41:05 on dist-test-slave-7c35"
server_key: "657d5584a6d35bb2e2c706a469e4f645"
server_key_iv: "2c6a6f54ddc22da6cc2cbcaf850ea345"
server_key_version: "encryptionkey@0"
I20260806 11:41:05.257997 12017 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786016169045816-21563-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786016169045816-21563-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786016169045816-21563-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:41:05.263310 12017 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:41:05.266966 12017 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:41:05.267199 12017 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:41:05.275075 12017 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45329
I20260806 11:41:05.275122 12086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45329 every 8 connection(s)
I20260806 11:41:05.276153 12017 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
I20260806 11:41:05.279374 12087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:41:05.279933 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 12017
I20260806 11:41:05.280086 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786016169045816-21563-0/minicluster-data/master-0/wal/instance
I20260806 11:41:05.280421 21563 external_mini_cluster.cc:1428] Setting key 4f577fae8cf97198c8ed2c8e43cedc6f
I20260806 11:41:05.286012 12090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:05.281945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.41:54707 (local address 127.0.0.1:45329)
0806 11:41:05.282408 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:05.282423 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:05.282460 (+ 37us) server_negotiation.cc:407] Connection header received
0806 11:41:05.283239 (+ 779us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:05.283259 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:05.283556 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:05.283906 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:05.284770 (+ 864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:41:05.284775 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:41:05.284788 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 11:41:05.284798 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:41:05.284853 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 11:41:05.284873 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:41:05.285030 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 11:41:05.285109 (+ 79us) server_negotiation.cc:299] Negotiation successful
0806 11:41:05.285279 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":132,"threads_started":1}
I20260806 11:41:05.287194 12087 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 709d1c47063348c3bd79bea0732d7add: Bootstrap starting.
I20260806 11:41:05.289407 21563 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 11:41:05.290113 12087 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 709d1c47063348c3bd79bea0732d7add: Neither blocks nor log segments found. Creating new log.
I20260806 11:41:05.291071 12087 log.cc:824] T 00000000000000000000000000000000 P 709d1c47063348c3bd79bea0732d7add: Log is configured to *not* fsync() on all Append() calls
I20260806 11:41:05.293792 12087 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 709d1c47063348c3bd79bea0732d7add: No bootstrap required, opened a new log
I20260806 11:41:05.294219 12090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:05.292138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.41:44669 (local address 127.0.0.1:45329)
0806 11:41:05.292288 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:05.292294 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:05.292571 (+ 277us) server_negotiation.cc:407] Connection header received
0806 11:41:05.292731 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:05.292734 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:05.292770 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:05.292844 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:05.293747 (+ 903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:41:05.293749 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:41:05.293752 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 11:41:05.293771 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:41:05.293799 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 11:41:05.293810 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:41:05.293964 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0806 11:41:05.294032 (+ 68us) server_negotiation.cc:299] Negotiation successful
0806 11:41:05.294071 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 11:41:05.298673 12087 raft_consensus.cc:348] T 00000000000000000000000000000000 P 709d1c47063348c3bd79bea0732d7add [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "709d1c47063348c3bd79bea0732d7add" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45329 } }
I20260806 11:41:05.299042 12087 raft_consensus.cc:374] T 00000000000000000000000000000000 P 709d1c47063348c3bd79bea0732d7add [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:41:05.299136 12087 raft_consensus.cc:729] T 00000000000000000000000000000000 P 709d1c47063348c3bd79bea0732d7add [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 709d1c47063348c3bd79bea0732d7add, State: Initialized, Role: FOLLOWER
I20260806 11:41:05.299664 12087 consensus_queue.cc:260] T 00000000000000000000000000000000 P 709d1c47063348c3bd79bea0732d7add [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "709d1c47063348c3bd79bea0732d7add" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45329 } }
I20260806 11:41:05.299825 12087 raft_consensus.cc:388] T 00000000000000000000000000000000 P 709d1c47063348c3bd79bea0732d7add [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 11:41:05.299898 12087 raft_consensus.cc:482] T 00000000000000000000000000000000 P 709d1c47063348c3bd79bea0732d7add [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 11:41:05.300011 12087 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 709d1c47063348c3bd79bea0732d7add [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:41:05.301134 12087 raft_consensus.cc:504] T 00000000000000000000000000000000 P 709d1c47063348c3bd79bea0732d7add [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "709d1c47063348c3bd79bea0732d7add" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45329 } }
I20260806 11:41:05.301584 12087 leader_election.cc:302] T 00000000000000000000000000000000 P 709d1c47063348c3bd79bea0732d7add [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 709d1c47063348c3bd79bea0732d7add; no voters:
I20260806 11:41:05.301952 12087 leader_election.cc:288] T 00000000000000000000000000000000 P 709d1c47063348c3bd79bea0732d7add [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 11:41:05.302127 12097 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 709d1c47063348c3bd79bea0732d7add [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:41:05.302397 12097 raft_consensus.cc:686] T 00000000000000000000000000000000 P 709d1c47063348c3bd79bea0732d7add [term 1 LEADER]: Becoming Leader. State: Replica: 709d1c47063348c3bd79bea0732d7add, State: Running, Role: LEADER
I20260806 11:41:05.302737 12097 consensus_queue.cc:237] T 00000000000000000000000000000000 P 709d1c47063348c3bd79bea0732d7add [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "709d1c47063348c3bd79bea0732d7add" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45329 } }
I20260806 11:41:05.303392 12087 sys_catalog.cc:564] T 00000000000000000000000000000000 P 709d1c47063348c3bd79bea0732d7add [sys.catalog]: configured and running, proceeding with master startup.
I20260806 11:41:05.304699 12100 sys_catalog.cc:455] T 00000000000000000000000000000000 P 709d1c47063348c3bd79bea0732d7add [sys.catalog]: SysCatalogTable state changed. Reason: New leader 709d1c47063348c3bd79bea0732d7add. Latest consensus state: current_term: 1 leader_uuid: "709d1c47063348c3bd79bea0732d7add" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "709d1c47063348c3bd79bea0732d7add" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45329 } } }
I20260806 11:41:05.304699 12098 sys_catalog.cc:455] T 00000000000000000000000000000000 P 709d1c47063348c3bd79bea0732d7add [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "709d1c47063348c3bd79bea0732d7add" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "709d1c47063348c3bd79bea0732d7add" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45329 } } }
I20260806 11:41:05.304908 12098 sys_catalog.cc:458] T 00000000000000000000000000000000 P 709d1c47063348c3bd79bea0732d7add [sys.catalog]: This master's current role is: LEADER
I20260806 11:41:05.304966 12100 sys_catalog.cc:458] T 00000000000000000000000000000000 P 709d1c47063348c3bd79bea0732d7add [sys.catalog]: This master's current role is: LEADER
I20260806 11:41:05.305298 12107 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 11:41:05.308761 12107 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 11:41:05.315197 12107 catalog_manager.cc:1347] Generated new cluster ID: f7a6e9867907442eb0fdf43df8042db9
I20260806 11:41:05.315299 12107 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 11:41:05.338122 12107 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 11:41:05.338514 12107 catalog_manager.cc:1504] Loading token signing keys...
I20260806 11:41:05.344158 12107 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 709d1c47063348c3bd79bea0732d7add: Generated new TSK 0
I20260806 11:41:05.344990 12107 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 11:41:05.357244 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 12017
2026-08-06T11:41:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1493 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786016169045816-21563-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 11:41:05 dist-test-slave-7c35 krb5kdc[12120](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 11:41:05 dist-test-slave-7c35 krb5kdc[12120](info): set up 2 sockets
Aug 06 11:41:05 dist-test-slave-7c35 krb5kdc[12120](info): commencing operation
krb5kdc: starting...
W20260806 11:41:06.434353 21563 mini_kdc.cc:121] Time spent starting KDC: real 1.055s user 0.005s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Aug 06 11:41:06 dist-test-slave-7c35 krb5kdc[12120](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016466, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T11:41:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T11:41:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.14.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab.
Entry for principal kudu/127.21.14.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab.
I20260806 11:41:06.578641 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786016169045816-21563-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786016169045816-21563-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786016169045816-21563-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:35749
--webserver_interface=127.21.14.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab
--principal=kudu/127.21.14.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:43565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.14.254:35749
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:41:06.701062 12133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:41:06.701413 12133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:41:06.701498 12133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:41:06.705765 12133 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 11:41:06.705899 12133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:41:06.705940 12133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:41:06.705969 12133 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 11:41:06.705993 12133 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 11:41:06.711593 12133 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:43565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786016169045816-21563-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786016169045816-21563-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.14.254:35749
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab
--principal=kudu/127.21.14.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:35749
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.14.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12133
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786016169045816-21563-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:41:06.712719 12133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:41:06.713778 12133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:41:06.723048 12138 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:41:06.723042 12142 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:41:06.723104 12133 server_base.cc:1034] running on GCE node
W20260806 11:41:06.723048 12139 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:41:06.723943 12133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:41:06.725206 12133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:41:06.726413 12133 hybrid_clock.cc:648] HybridClock initialized: now 1786016466726398 us; error 28 us; skew 500 ppm
Aug 06 11:41:06 dist-test-slave-7c35 krb5kdc[12120](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016466, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:41:06.729456 12133 init.cc:377] Logged in from keytab as kudu/127.21.14.254@KRBTEST.COM (short username kudu)
I20260806 11:41:06.730468 12133 webserver.cc:460] Webserver started at http://127.21.14.254:41509/ using document root <none> and password file <none>
I20260806 11:41:06.731163 12133 fs_manager.cc:359] Metadata directory not provided
I20260806 11:41:06.731259 12133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:41:06.731513 12133 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:41:06.733704 12133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786016169045816-21563-0/minicluster-data/master-0/data/instance:
uuid: "f76ba8d3ca0846618a452d518c8f156a"
format_stamp: "Formatted at 2026-08-06 11:41:06 on dist-test-slave-7c35"
server_key: "402c396ce1d63769f4122fb2e59bf629"
server_key_iv: "7c632237992d811d80355891f1b96797"
server_key_version: "encryptionkey@0"
I20260806 11:41:06.734313 12133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786016169045816-21563-0/minicluster-data/master-0/wal/instance:
uuid: "f76ba8d3ca0846618a452d518c8f156a"
format_stamp: "Formatted at 2026-08-06 11:41:06 on dist-test-slave-7c35"
server_key: "402c396ce1d63769f4122fb2e59bf629"
server_key_iv: "7c632237992d811d80355891f1b96797"
server_key_version: "encryptionkey@0"
I20260806 11:41:06.738479 12133 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 11:41:06.741371 12149 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:41:06.742833 12133 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 11:41:06.743088 12133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786016169045816-21563-0/minicluster-data/master-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786016169045816-21563-0/minicluster-data/master-0/wal
uuid: "f76ba8d3ca0846618a452d518c8f156a"
format_stamp: "Formatted at 2026-08-06 11:41:06 on dist-test-slave-7c35"
server_key: "402c396ce1d63769f4122fb2e59bf629"
server_key_iv: "7c632237992d811d80355891f1b96797"
server_key_version: "encryptionkey@0"
I20260806 11:41:06.743222 12133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786016169045816-21563-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786016169045816-21563-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786016169045816-21563-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:41:06.761794 12133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:41:06.765744 12133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:41:06.766004 12133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:41:06.774286 12133 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.254:35749
I20260806 11:41:06.774302 12203 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.254:35749 every 8 connection(s)
I20260806 11:41:06.775377 12133 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
I20260806 11:41:06.777403 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 12133
I20260806 11:41:06.777571 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786016169045816-21563-0/minicluster-data/master-0/wal/instance
I20260806 11:41:06.777886 21563 external_mini_cluster.cc:1428] Setting key 6a061346cbfc1d43de380598cfb1dc03
I20260806 11:41:06.778527 12204 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:41:06.785905 12204 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f76ba8d3ca0846618a452d518c8f156a: Bootstrap starting.
Aug 06 11:41:06 dist-test-slave-7c35 krb5kdc[12120](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016466, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:41:06.788734 12204 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f76ba8d3ca0846618a452d518c8f156a: Neither blocks nor log segments found. Creating new log.
I20260806 11:41:06.789772 12204 log.cc:824] T 00000000000000000000000000000000 P f76ba8d3ca0846618a452d518c8f156a: Log is configured to *not* fsync() on all Append() calls
I20260806 11:41:06.792591 12204 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f76ba8d3ca0846618a452d518c8f156a: No bootstrap required, opened a new log
I20260806 11:41:06.795033 12207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:06.779326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50225 (local address 127.21.14.254:35749)
0806 11:41:06.779794 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:06.779814 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:06.779858 (+ 44us) server_negotiation.cc:407] Connection header received
0806 11:41:06.780750 (+ 892us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:06.780776 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:06.781246 (+ 470us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:06.781645 (+ 399us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:06.782368 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:06.783906 (+ 1538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:06.784880 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:06.785597 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:06.785766 (+ 169us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:41:06.788858 (+ 3092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:41:06.788865 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:41:06.788886 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:41:06.788926 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:41:06.791128 (+ 2202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:41:06.791764 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:41:06.791770 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:41:06.791776 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:41:06.791924 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:41:06.792331 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:41:06.792335 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:41:06.792337 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:41:06.792888 (+ 551us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:41:06.793121 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0806 11:41:06.793429 (+ 308us) server_negotiation.cc:299] Negotiation successful
0806 11:41:06.793679 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":137,"threads_started":1}
I20260806 11:41:06.796891 12204 raft_consensus.cc:348] T 00000000000000000000000000000000 P f76ba8d3ca0846618a452d518c8f156a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f76ba8d3ca0846618a452d518c8f156a" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 35749 } }
I20260806 11:41:06.797222 12204 raft_consensus.cc:374] T 00000000000000000000000000000000 P f76ba8d3ca0846618a452d518c8f156a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:41:06.797312 12204 raft_consensus.cc:729] T 00000000000000000000000000000000 P f76ba8d3ca0846618a452d518c8f156a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f76ba8d3ca0846618a452d518c8f156a, State: Initialized, Role: FOLLOWER
I20260806 11:41:06.797817 12204 consensus_queue.cc:260] T 00000000000000000000000000000000 P f76ba8d3ca0846618a452d518c8f156a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f76ba8d3ca0846618a452d518c8f156a" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 35749 } }
I20260806 11:41:06.797963 12204 raft_consensus.cc:388] T 00000000000000000000000000000000 P f76ba8d3ca0846618a452d518c8f156a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 11:41:06.798055 12204 raft_consensus.cc:482] T 00000000000000000000000000000000 P f76ba8d3ca0846618a452d518c8f156a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 11:41:06.798163 12204 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f76ba8d3ca0846618a452d518c8f156a [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:41:06.798658 21563 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 11:41:06.799527 12204 raft_consensus.cc:504] T 00000000000000000000000000000000 P f76ba8d3ca0846618a452d518c8f156a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f76ba8d3ca0846618a452d518c8f156a" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 35749 } }
I20260806 11:41:06.799932 12204 leader_election.cc:302] T 00000000000000000000000000000000 P f76ba8d3ca0846618a452d518c8f156a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f76ba8d3ca0846618a452d518c8f156a; no voters:
I20260806 11:41:06.800398 12204 leader_election.cc:288] T 00000000000000000000000000000000 P f76ba8d3ca0846618a452d518c8f156a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 11:41:06.800549 12213 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f76ba8d3ca0846618a452d518c8f156a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:41:06.800832 12213 raft_consensus.cc:686] T 00000000000000000000000000000000 P f76ba8d3ca0846618a452d518c8f156a [term 1 LEADER]: Becoming Leader. State: Replica: f76ba8d3ca0846618a452d518c8f156a, State: Running, Role: LEADER
I20260806 11:41:06.801299 12213 consensus_queue.cc:237] T 00000000000000000000000000000000 P f76ba8d3ca0846618a452d518c8f156a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f76ba8d3ca0846618a452d518c8f156a" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 35749 } }
I20260806 11:41:06.801901 12204 sys_catalog.cc:564] T 00000000000000000000000000000000 P f76ba8d3ca0846618a452d518c8f156a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 11:41:06.803263 12214 sys_catalog.cc:455] T 00000000000000000000000000000000 P f76ba8d3ca0846618a452d518c8f156a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f76ba8d3ca0846618a452d518c8f156a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f76ba8d3ca0846618a452d518c8f156a" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 35749 } } }
I20260806 11:41:06.803478 12214 sys_catalog.cc:458] T 00000000000000000000000000000000 P f76ba8d3ca0846618a452d518c8f156a [sys.catalog]: This master's current role is: LEADER
I20260806 11:41:06.803785 12214 sys_catalog.cc:455] T 00000000000000000000000000000000 P f76ba8d3ca0846618a452d518c8f156a [sys.catalog]: SysCatalogTable state changed. Reason: New leader f76ba8d3ca0846618a452d518c8f156a. Latest consensus state: current_term: 1 leader_uuid: "f76ba8d3ca0846618a452d518c8f156a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f76ba8d3ca0846618a452d518c8f156a" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 35749 } } }
I20260806 11:41:06.803970 12214 sys_catalog.cc:458] T 00000000000000000000000000000000 P f76ba8d3ca0846618a452d518c8f156a [sys.catalog]: This master's current role is: LEADER
W20260806 11:41:06.807612 12229 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f76ba8d3ca0846618a452d518c8f156a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 11:41:06.807703 12229 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 11:41:06.807847 12225 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 11:41:06.808894 12225 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 11:41:06.811293 12207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:06.801473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36589 (local address 127.21.14.254:35749)
0806 11:41:06.801630 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:06.801637 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:06.801654 (+ 17us) server_negotiation.cc:407] Connection header received
0806 11:41:06.801696 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:06.801699 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:06.801747 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:06.801841 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:06.802932 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:06.803764 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:06.805037 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:06.805289 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:06.805384 (+ 95us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:41:06.807259 (+ 1875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:41:06.807263 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:41:06.807266 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:41:06.807293 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:41:06.809147 (+ 1854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:41:06.809774 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:41:06.809777 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:41:06.809779 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:41:06.809865 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:41:06.810305 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:41:06.810309 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:41:06.810312 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:41:06.810536 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:41:06.810688 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0806 11:41:06.811042 (+ 354us) server_negotiation.cc:299] Negotiation successful
0806 11:41:06.811142 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 11:41:06.815146 12225 catalog_manager.cc:1347] Generated new cluster ID: 9410ffe1857a429387beae7c007bc17c
I20260806 11:41:06.815220 12225 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 11:41:06.830015 12225 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 11:41:06.831480 12225 catalog_manager.cc:1504] Loading token signing keys...
I20260806 11:41:06.839296 12225 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f76ba8d3ca0846618a452d518c8f156a: Generated new TSK 0
I20260806 11:41:06.840233 12225 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 11:41:06.846487 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 12133
W20260806 11:41:06.855194 12118 connection.cc:535] client connection to 127.21.14.254:35749 recv error: Network error: failed to read from TLS socket (remote: 127.21.14.254:35749): Connection reset by peer (error 104)
2026-08-06T11:41:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1490 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786016169045816-21563-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 11:41:06 dist-test-slave-7c35 krb5kdc[12235](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 11:41:06 dist-test-slave-7c35 krb5kdc[12235](info): set up 2 sockets
Aug 06 11:41:06 dist-test-slave-7c35 krb5kdc[12235](info): commencing operation
krb5kdc: starting...
W20260806 11:41:07.944972 21563 mini_kdc.cc:121] Time spent starting KDC: real 1.074s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Aug 06 11:41:08 dist-test-slave-7c35 krb5kdc[12235](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016468, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T11:41:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T11:41:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.14.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab.
Entry for principal kudu/127.21.14.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab.
I20260806 11:41:08.090781 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786016169045816-21563-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786016169045816-21563-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786016169045816-21563-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:37299
--webserver_interface=127.21.14.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab
--principal=kudu/127.21.14.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:44877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.14.254:37299
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:41:08.211122 12248 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:41:08.211443 12248 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:41:08.211490 12248 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:41:08.215616 12248 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 11:41:08.215696 12248 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:41:08.215718 12248 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:41:08.215735 12248 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 11:41:08.215752 12248 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 11:41:08.222012 12248 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:44877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786016169045816-21563-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786016169045816-21563-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.14.254:37299
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab
--principal=kudu/127.21.14.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:37299
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.14.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12248
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786016169045816-21563-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:41:08.223268 12248 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:41:08.224359 12248 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:41:08.233983 12253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:41:08.233973 12256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:41:08.234066 12248 server_base.cc:1034] running on GCE node
W20260806 11:41:08.233974 12254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:41:08.234969 12248 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:41:08.236196 12248 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:41:08.237428 12248 hybrid_clock.cc:648] HybridClock initialized: now 1786016468237395 us; error 46 us; skew 500 ppm
Aug 06 11:41:08 dist-test-slave-7c35 krb5kdc[12235](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016468, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:41:08.240569 12248 init.cc:377] Logged in from keytab as kudu/127.21.14.254@KRBTEST.COM (short username kudu)
I20260806 11:41:08.241638 12248 webserver.cc:460] Webserver started at http://127.21.14.254:38555/ using document root <none> and password file <none>
I20260806 11:41:08.242259 12248 fs_manager.cc:359] Metadata directory not provided
I20260806 11:41:08.242358 12248 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:41:08.242622 12248 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:41:08.244897 12248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786016169045816-21563-0/minicluster-data/master-0/data/instance:
uuid: "2d69e91521e04a6a90595c3fc30360b3"
format_stamp: "Formatted at 2026-08-06 11:41:08 on dist-test-slave-7c35"
server_key: "324ebd174c7ace400105792bb02df25c"
server_key_iv: "4f9bb50af706c6267cdbe5e2295d4c4d"
server_key_version: "encryptionkey@0"
I20260806 11:41:08.245550 12248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786016169045816-21563-0/minicluster-data/master-0/wal/instance:
uuid: "2d69e91521e04a6a90595c3fc30360b3"
format_stamp: "Formatted at 2026-08-06 11:41:08 on dist-test-slave-7c35"
server_key: "324ebd174c7ace400105792bb02df25c"
server_key_iv: "4f9bb50af706c6267cdbe5e2295d4c4d"
server_key_version: "encryptionkey@0"
I20260806 11:41:08.249790 12248 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 11:41:08.252743 12264 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:41:08.254277 12248 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 11:41:08.254457 12248 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786016169045816-21563-0/minicluster-data/master-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786016169045816-21563-0/minicluster-data/master-0/wal
uuid: "2d69e91521e04a6a90595c3fc30360b3"
format_stamp: "Formatted at 2026-08-06 11:41:08 on dist-test-slave-7c35"
server_key: "324ebd174c7ace400105792bb02df25c"
server_key_iv: "4f9bb50af706c6267cdbe5e2295d4c4d"
server_key_version: "encryptionkey@0"
I20260806 11:41:08.254602 12248 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786016169045816-21563-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786016169045816-21563-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786016169045816-21563-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:41:08.271405 12248 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:41:08.275341 12248 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:41:08.275585 12248 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:41:08.283341 12248 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.254:37299
I20260806 11:41:08.283370 12318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.254:37299 every 8 connection(s)
I20260806 11:41:08.284387 12248 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
I20260806 11:41:08.287438 12319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:41:08.289295 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 12248
I20260806 11:41:08.289458 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786016169045816-21563-0/minicluster-data/master-0/wal/instance
I20260806 11:41:08.289770 21563 external_mini_cluster.cc:1428] Setting key 1864973d6650e46a2b2f53019a07d876
I20260806 11:41:08.294824 12319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d69e91521e04a6a90595c3fc30360b3: Bootstrap starting.
I20260806 11:41:08.297601 12319 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d69e91521e04a6a90595c3fc30360b3: Neither blocks nor log segments found. Creating new log.
I20260806 11:41:08.298589 12319 log.cc:824] T 00000000000000000000000000000000 P 2d69e91521e04a6a90595c3fc30360b3: Log is configured to *not* fsync() on all Append() calls
I20260806 11:41:08.300593 12322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:08.291341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36663 (local address 127.21.14.254:37299)
0806 11:41:08.291844 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:08.291861 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:08.291905 (+ 44us) server_negotiation.cc:407] Connection header received
0806 11:41:08.292761 (+ 856us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:08.292789 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:08.293241 (+ 452us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:08.293644 (+ 403us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:08.294088 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:08.295605 (+ 1517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:08.296560 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:08.297218 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:08.297432 (+ 214us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:41:08.298428 (+ 996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:41:08.298436 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:41:08.298456 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 11:41:08.299012 (+ 556us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:41:08.299089 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 11:41:08.299128 (+ 39us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:41:08.299282 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0806 11:41:08.299483 (+ 201us) server_negotiation.cc:299] Negotiation successful
0806 11:41:08.299796 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":151,"threads_started":1}
I20260806 11:41:08.301009 12319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d69e91521e04a6a90595c3fc30360b3: No bootstrap required, opened a new log
I20260806 11:41:08.304049 21563 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 11:41:08.305414 12319 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d69e91521e04a6a90595c3fc30360b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d69e91521e04a6a90595c3fc30360b3" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 37299 } }
I20260806 11:41:08.305735 12319 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2d69e91521e04a6a90595c3fc30360b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:41:08.305817 12319 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d69e91521e04a6a90595c3fc30360b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d69e91521e04a6a90595c3fc30360b3, State: Initialized, Role: FOLLOWER
I20260806 11:41:08.306381 12319 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d69e91521e04a6a90595c3fc30360b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d69e91521e04a6a90595c3fc30360b3" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 37299 } }
I20260806 11:41:08.306659 12319 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2d69e91521e04a6a90595c3fc30360b3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 11:41:08.306759 12319 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d69e91521e04a6a90595c3fc30360b3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 11:41:08.306901 12319 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d69e91521e04a6a90595c3fc30360b3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:41:08.307991 12319 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d69e91521e04a6a90595c3fc30360b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d69e91521e04a6a90595c3fc30360b3" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 37299 } }
I20260806 11:41:08.308383 12319 leader_election.cc:302] T 00000000000000000000000000000000 P 2d69e91521e04a6a90595c3fc30360b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d69e91521e04a6a90595c3fc30360b3; no voters:
I20260806 11:41:08.308735 12319 leader_election.cc:288] T 00000000000000000000000000000000 P 2d69e91521e04a6a90595c3fc30360b3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 11:41:08.308861 12329 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d69e91521e04a6a90595c3fc30360b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:41:08.309103 12329 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d69e91521e04a6a90595c3fc30360b3 [term 1 LEADER]: Becoming Leader. State: Replica: 2d69e91521e04a6a90595c3fc30360b3, State: Running, Role: LEADER
I20260806 11:41:08.309531 12329 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d69e91521e04a6a90595c3fc30360b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d69e91521e04a6a90595c3fc30360b3" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 37299 } }
I20260806 11:41:08.310235 12319 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d69e91521e04a6a90595c3fc30360b3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 11:41:08.311367 12322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:08.306375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50671 (local address 127.21.14.254:37299)
0806 11:41:08.306530 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:08.306535 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:08.306711 (+ 176us) server_negotiation.cc:407] Connection header received
0806 11:41:08.306907 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:08.306910 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:08.306964 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:08.307056 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:08.307367 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:08.307951 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:08.308977 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:08.309350 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:08.309432 (+ 82us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:41:08.310679 (+ 1247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:41:08.310683 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:41:08.310686 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 11:41:08.310700 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:41:08.310732 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 11:41:08.310746 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:41:08.310839 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 11:41:08.311126 (+ 287us) server_negotiation.cc:299] Negotiation successful
0806 11:41:08.311178 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 11:41:08.311537 12330 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d69e91521e04a6a90595c3fc30360b3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2d69e91521e04a6a90595c3fc30360b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d69e91521e04a6a90595c3fc30360b3" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 37299 } } }
I20260806 11:41:08.311676 12330 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d69e91521e04a6a90595c3fc30360b3 [sys.catalog]: This master's current role is: LEADER
I20260806 11:41:08.312081 12331 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d69e91521e04a6a90595c3fc30360b3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d69e91521e04a6a90595c3fc30360b3. Latest consensus state: current_term: 1 leader_uuid: "2d69e91521e04a6a90595c3fc30360b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d69e91521e04a6a90595c3fc30360b3" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 37299 } } }
I20260806 11:41:08.312233 12331 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d69e91521e04a6a90595c3fc30360b3 [sys.catalog]: This master's current role is: LEADER
I20260806 11:41:08.312601 12338 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 11:41:08.315680 12338 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 11:41:08.321926 12338 catalog_manager.cc:1347] Generated new cluster ID: 884ddbb3ebe44aadaadb4d385949e002
I20260806 11:41:08.322019 12338 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 11:41:08.327668 12338 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 11:41:08.328666 12338 catalog_manager.cc:1504] Loading token signing keys...
I20260806 11:41:08.345813 12338 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2d69e91521e04a6a90595c3fc30360b3: Generated new TSK 0
I20260806 11:41:08.346907 12338 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 11:41:08.360702 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 12248
W20260806 11:41:08.368600 12233 connection.cc:535] client connection to 127.21.14.254:37299 recv error: Network error: failed to read from TLS socket (remote: 127.21.14.254:37299): Connection reset by peer (error 104)
2026-08-06T11:41:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1513 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786016169045816-21563-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 11:41:08 dist-test-slave-7c35 krb5kdc[12350](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 11:41:08 dist-test-slave-7c35 krb5kdc[12350](info): set up 2 sockets
Aug 06 11:41:08 dist-test-slave-7c35 krb5kdc[12350](info): commencing operation
krb5kdc: starting...
W20260806 11:41:09.455343 21563 mini_kdc.cc:121] Time spent starting KDC: real 1.072s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Aug 06 11:41:09 dist-test-slave-7c35 krb5kdc[12350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016469, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T11:41:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T11:41:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.14.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab.
Entry for principal kudu/127.21.14.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab.
I20260806 11:41:09.611709 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786016169045816-21563-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786016169045816-21563-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786016169045816-21563-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:44167
--webserver_interface=127.21.14.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab
--principal=kudu/127.21.14.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:37117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.14.254:44167
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:41:09.737943 12364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:41:09.738257 12364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:41:09.738301 12364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:41:09.742458 12364 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 11:41:09.742544 12364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:41:09.742565 12364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:41:09.742583 12364 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 11:41:09.742599 12364 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 11:41:09.749063 12364 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:37117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786016169045816-21563-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786016169045816-21563-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.14.254:44167
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab
--principal=kudu/127.21.14.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:44167
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.14.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12364
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786016169045816-21563-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:41:09.750371 12364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:41:09.751590 12364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:41:09.761196 12369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:41:09.761196 12372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:41:09.761257 12364 server_base.cc:1034] running on GCE node
W20260806 11:41:09.761198 12370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:41:09.762197 12364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:41:09.763572 12364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:41:09.764802 12364 hybrid_clock.cc:648] HybridClock initialized: now 1786016469764763 us; error 54 us; skew 500 ppm
Aug 06 11:41:09 dist-test-slave-7c35 krb5kdc[12350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016469, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:41:09.772692 12364 init.cc:377] Logged in from keytab as kudu/127.21.14.254@KRBTEST.COM (short username kudu)
I20260806 11:41:09.773805 12364 webserver.cc:460] Webserver started at http://127.21.14.254:39895/ using document root <none> and password file <none>
I20260806 11:41:09.774449 12364 fs_manager.cc:359] Metadata directory not provided
I20260806 11:41:09.774550 12364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:41:09.774813 12364 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:41:09.777087 12364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786016169045816-21563-0/minicluster-data/master-0/data/instance:
uuid: "7ab1c6b23a7445f691ba546e13e6f225"
format_stamp: "Formatted at 2026-08-06 11:41:09 on dist-test-slave-7c35"
server_key: "62873444778d93dd1ca405262418f490"
server_key_iv: "d2f50301239d2d7e0c94c9c973240e06"
server_key_version: "encryptionkey@0"
I20260806 11:41:09.777704 12364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786016169045816-21563-0/minicluster-data/master-0/wal/instance:
uuid: "7ab1c6b23a7445f691ba546e13e6f225"
format_stamp: "Formatted at 2026-08-06 11:41:09 on dist-test-slave-7c35"
server_key: "62873444778d93dd1ca405262418f490"
server_key_iv: "d2f50301239d2d7e0c94c9c973240e06"
server_key_version: "encryptionkey@0"
I20260806 11:41:09.782027 12364 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260806 11:41:09.784757 12379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:41:09.786084 12364 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 11:41:09.786240 12364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786016169045816-21563-0/minicluster-data/master-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786016169045816-21563-0/minicluster-data/master-0/wal
uuid: "7ab1c6b23a7445f691ba546e13e6f225"
format_stamp: "Formatted at 2026-08-06 11:41:09 on dist-test-slave-7c35"
server_key: "62873444778d93dd1ca405262418f490"
server_key_iv: "d2f50301239d2d7e0c94c9c973240e06"
server_key_version: "encryptionkey@0"
I20260806 11:41:09.786368 12364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786016169045816-21563-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786016169045816-21563-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786016169045816-21563-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:41:09.791384 12364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:41:09.794595 12364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:41:09.794809 12364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:41:09.802570 12364 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.254:44167
I20260806 11:41:09.802610 12431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.254:44167 every 8 connection(s)
I20260806 11:41:09.803669 12364 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
I20260806 11:41:09.806545 12432 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:41:09.810398 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 12364
I20260806 11:41:09.810596 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786016169045816-21563-0/minicluster-data/master-0/wal/instance
I20260806 11:41:09.810956 21563 external_mini_cluster.cc:1428] Setting key 48ad1e6e5da7b9f7368e2f0c0e32deba
I20260806 11:41:09.812928 12432 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ab1c6b23a7445f691ba546e13e6f225: Bootstrap starting.
I20260806 11:41:09.815429 12432 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7ab1c6b23a7445f691ba546e13e6f225: Neither blocks nor log segments found. Creating new log.
I20260806 11:41:09.816241 12432 log.cc:824] T 00000000000000000000000000000000 P 7ab1c6b23a7445f691ba546e13e6f225: Log is configured to *not* fsync() on all Append() calls
I20260806 11:41:09.816433 12435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:09.812435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52867 (local address 127.21.14.254:44167)
0806 11:41:09.812915 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:09.812931 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:09.812967 (+ 36us) server_negotiation.cc:407] Connection header received
0806 11:41:09.813654 (+ 687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:09.813672 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:09.813963 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:09.814317 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:09.815139 (+ 822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:41:09.815143 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:41:09.815156 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 11:41:09.815166 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:41:09.815219 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 11:41:09.815239 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:41:09.815359 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 11:41:09.815467 (+ 108us) server_negotiation.cc:299] Negotiation successful
0806 11:41:09.815643 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":155,"threads_started":1}
I20260806 11:41:09.818539 12432 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ab1c6b23a7445f691ba546e13e6f225: No bootstrap required, opened a new log
I20260806 11:41:09.819439 21563 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 11:41:09.822907 12432 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7ab1c6b23a7445f691ba546e13e6f225 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ab1c6b23a7445f691ba546e13e6f225" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 44167 } }
I20260806 11:41:09.823231 12432 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7ab1c6b23a7445f691ba546e13e6f225 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:41:09.823328 12432 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7ab1c6b23a7445f691ba546e13e6f225 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ab1c6b23a7445f691ba546e13e6f225, State: Initialized, Role: FOLLOWER
I20260806 11:41:09.823669 12435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:09.821672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57173 (local address 127.21.14.254:44167)
0806 11:41:09.821817 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:09.821824 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:09.822032 (+ 208us) server_negotiation.cc:407] Connection header received
0806 11:41:09.822215 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:09.822218 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:09.822261 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:09.822342 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:09.823163 (+ 821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:41:09.823166 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:41:09.823169 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 11:41:09.823181 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:41:09.823209 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 11:41:09.823227 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:41:09.823334 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 11:41:09.823468 (+ 134us) server_negotiation.cc:299] Negotiation successful
0806 11:41:09.823515 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 11:41:09.823907 12432 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7ab1c6b23a7445f691ba546e13e6f225 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ab1c6b23a7445f691ba546e13e6f225" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 44167 } }
I20260806 11:41:09.824101 12432 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7ab1c6b23a7445f691ba546e13e6f225 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 11:41:09.824198 12432 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7ab1c6b23a7445f691ba546e13e6f225 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 11:41:09.824309 12432 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7ab1c6b23a7445f691ba546e13e6f225 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:41:09.825464 12432 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7ab1c6b23a7445f691ba546e13e6f225 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ab1c6b23a7445f691ba546e13e6f225" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 44167 } }
I20260806 11:41:09.825842 12432 leader_election.cc:302] T 00000000000000000000000000000000 P 7ab1c6b23a7445f691ba546e13e6f225 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7ab1c6b23a7445f691ba546e13e6f225; no voters:
I20260806 11:41:09.826233 12432 leader_election.cc:288] T 00000000000000000000000000000000 P 7ab1c6b23a7445f691ba546e13e6f225 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 11:41:09.826380 12442 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7ab1c6b23a7445f691ba546e13e6f225 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:41:09.826617 12442 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7ab1c6b23a7445f691ba546e13e6f225 [term 1 LEADER]: Becoming Leader. State: Replica: 7ab1c6b23a7445f691ba546e13e6f225, State: Running, Role: LEADER
I20260806 11:41:09.827054 12442 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7ab1c6b23a7445f691ba546e13e6f225 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ab1c6b23a7445f691ba546e13e6f225" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 44167 } }
I20260806 11:41:09.827639 12432 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7ab1c6b23a7445f691ba546e13e6f225 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 11:41:09.828871 12444 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ab1c6b23a7445f691ba546e13e6f225 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7ab1c6b23a7445f691ba546e13e6f225" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ab1c6b23a7445f691ba546e13e6f225" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 44167 } } }
I20260806 11:41:09.829092 12444 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ab1c6b23a7445f691ba546e13e6f225 [sys.catalog]: This master's current role is: LEADER
I20260806 11:41:09.829538 12443 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ab1c6b23a7445f691ba546e13e6f225 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7ab1c6b23a7445f691ba546e13e6f225. Latest consensus state: current_term: 1 leader_uuid: "7ab1c6b23a7445f691ba546e13e6f225" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ab1c6b23a7445f691ba546e13e6f225" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 44167 } } }
I20260806 11:41:09.829638 12443 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ab1c6b23a7445f691ba546e13e6f225 [sys.catalog]: This master's current role is: LEADER
W20260806 11:41:09.833181 12458 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7ab1c6b23a7445f691ba546e13e6f225: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 11:41:09.833284 12458 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 11:41:09.833616 12453 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 11:41:09.834545 12453 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 11:41:09.840927 12453 catalog_manager.cc:1347] Generated new cluster ID: 26ce96630e2b410f92b1cdcb0a3f5233
I20260806 11:41:09.841022 12453 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 11:41:09.855079 12453 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 11:41:09.855526 12453 catalog_manager.cc:1504] Loading token signing keys...
I20260806 11:41:09.863273 12453 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7ab1c6b23a7445f691ba546e13e6f225: Generated new TSK 0
I20260806 11:41:09.864243 12453 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 11:41:09.873587 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 12364
2026-08-06T11:41:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1512 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786016169045816-21563-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 11:41:09 dist-test-slave-7c35 krb5kdc[12464](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 11:41:09 dist-test-slave-7c35 krb5kdc[12464](info): set up 2 sockets
Aug 06 11:41:09 dist-test-slave-7c35 krb5kdc[12464](info): commencing operation
krb5kdc: starting...
W20260806 11:41:10.967258 21563 mini_kdc.cc:121] Time spent starting KDC: real 1.071s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Aug 06 11:41:11 dist-test-slave-7c35 krb5kdc[12464](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016471, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T11:41:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T11:41:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.14.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab.
Entry for principal kudu/127.21.14.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab.
I20260806 11:41:11.107237 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786016169045816-21563-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786016169045816-21563-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786016169045816-21563-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:42481
--webserver_interface=127.21.14.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab
--principal=kudu/127.21.14.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:38359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.14.254:42481
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:41:11.229079 12478 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:41:11.229382 12478 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:41:11.229434 12478 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:41:11.233714 12478 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 11:41:11.233793 12478 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:41:11.233814 12478 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:41:11.233830 12478 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 11:41:11.233845 12478 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 11:41:11.239653 12478 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:38359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786016169045816-21563-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786016169045816-21563-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.14.254:42481
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab
--principal=kudu/127.21.14.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:42481
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.14.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12478
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786016169045816-21563-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:41:11.240863 12478 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:41:11.241976 12478 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:41:11.250898 12483 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:41:11.250907 12484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:41:11.250902 12486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:41:11.251693 12478 server_base.cc:1034] running on GCE node
I20260806 11:41:11.252338 12478 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:41:11.253662 12478 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:41:11.254911 12478 hybrid_clock.cc:648] HybridClock initialized: now 1786016471254889 us; error 37 us; skew 500 ppm
Aug 06 11:41:11 dist-test-slave-7c35 krb5kdc[12464](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016471, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:41:11.257946 12478 init.cc:377] Logged in from keytab as kudu/127.21.14.254@KRBTEST.COM (short username kudu)
I20260806 11:41:11.259014 12478 webserver.cc:460] Webserver started at http://127.21.14.254:41117/ using document root <none> and password file <none>
I20260806 11:41:11.259650 12478 fs_manager.cc:359] Metadata directory not provided
I20260806 11:41:11.259752 12478 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:41:11.260002 12478 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:41:11.262195 12478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786016169045816-21563-0/minicluster-data/master-0/data/instance:
uuid: "a5810eb7ebb54e919ddee57060134845"
format_stamp: "Formatted at 2026-08-06 11:41:11 on dist-test-slave-7c35"
server_key: "60b7936845f667034adef2392b392aee"
server_key_iv: "016e589989d8d5eaa17f1fb868b56ecd"
server_key_version: "encryptionkey@0"
I20260806 11:41:11.262812 12478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786016169045816-21563-0/minicluster-data/master-0/wal/instance:
uuid: "a5810eb7ebb54e919ddee57060134845"
format_stamp: "Formatted at 2026-08-06 11:41:11 on dist-test-slave-7c35"
server_key: "60b7936845f667034adef2392b392aee"
server_key_iv: "016e589989d8d5eaa17f1fb868b56ecd"
server_key_version: "encryptionkey@0"
I20260806 11:41:11.266995 12478 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 11:41:11.269651 12493 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:41:11.271054 12478 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 11:41:11.271201 12478 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786016169045816-21563-0/minicluster-data/master-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786016169045816-21563-0/minicluster-data/master-0/wal
uuid: "a5810eb7ebb54e919ddee57060134845"
format_stamp: "Formatted at 2026-08-06 11:41:11 on dist-test-slave-7c35"
server_key: "60b7936845f667034adef2392b392aee"
server_key_iv: "016e589989d8d5eaa17f1fb868b56ecd"
server_key_version: "encryptionkey@0"
I20260806 11:41:11.271327 12478 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786016169045816-21563-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786016169045816-21563-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786016169045816-21563-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:41:11.289908 12478 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:41:11.293547 12478 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:41:11.293784 12478 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:41:11.301488 12478 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.254:42481
I20260806 11:41:11.301571 12549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.254:42481 every 8 connection(s)
I20260806 11:41:11.302539 12478 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
I20260806 11:41:11.305481 12550 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:41:11.305617 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 12478
I20260806 11:41:11.305763 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786016169045816-21563-0/minicluster-data/master-0/wal/instance
I20260806 11:41:11.306058 21563 external_mini_cluster.cc:1428] Setting key 4a9db9426fdc4d2960f4d813011300c4
I20260806 11:41:11.313107 12550 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5810eb7ebb54e919ddee57060134845: Bootstrap starting.
Aug 06 11:41:11 dist-test-slave-7c35 krb5kdc[12464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016471, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:41:11.316171 12550 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a5810eb7ebb54e919ddee57060134845: Neither blocks nor log segments found. Creating new log.
I20260806 11:41:11.317003 12550 log.cc:824] T 00000000000000000000000000000000 P a5810eb7ebb54e919ddee57060134845: Log is configured to *not* fsync() on all Append() calls
I20260806 11:41:11.319285 12550 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5810eb7ebb54e919ddee57060134845: No bootstrap required, opened a new log
I20260806 11:41:11.321758 12553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:11.307612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42709 (local address 127.21.14.254:42481)
0806 11:41:11.308095 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:11.308113 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:11.308152 (+ 39us) server_negotiation.cc:407] Connection header received
0806 11:41:11.308956 (+ 804us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:11.308977 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:11.309388 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:11.309701 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:11.310783 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:11.311913 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:11.313152 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:11.313716 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:11.313849 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:41:11.316813 (+ 2964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:41:11.316819 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:41:11.316835 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:41:11.316869 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:41:11.318706 (+ 1837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:41:11.319252 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:41:11.319258 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:41:11.319262 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:41:11.319446 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:41:11.319760 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:41:11.319764 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:41:11.319765 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:41:11.320186 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:41:11.320403 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0806 11:41:11.320699 (+ 296us) server_negotiation.cc:299] Negotiation successful
0806 11:41:11.321003 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":181,"threads_started":1}
I20260806 11:41:11.323570 12550 raft_consensus.cc:348] T 00000000000000000000000000000000 P a5810eb7ebb54e919ddee57060134845 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5810eb7ebb54e919ddee57060134845" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 42481 } }
I20260806 11:41:11.323889 12550 raft_consensus.cc:374] T 00000000000000000000000000000000 P a5810eb7ebb54e919ddee57060134845 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:41:11.324002 12550 raft_consensus.cc:729] T 00000000000000000000000000000000 P a5810eb7ebb54e919ddee57060134845 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5810eb7ebb54e919ddee57060134845, State: Initialized, Role: FOLLOWER
I20260806 11:41:11.324507 12550 consensus_queue.cc:260] T 00000000000000000000000000000000 P a5810eb7ebb54e919ddee57060134845 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5810eb7ebb54e919ddee57060134845" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 42481 } }
I20260806 11:41:11.324640 12550 raft_consensus.cc:388] T 00000000000000000000000000000000 P a5810eb7ebb54e919ddee57060134845 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 11:41:11.324726 12550 raft_consensus.cc:482] T 00000000000000000000000000000000 P a5810eb7ebb54e919ddee57060134845 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 11:41:11.324864 12550 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a5810eb7ebb54e919ddee57060134845 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:41:11.325053 21563 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 11:41:11.325929 12550 raft_consensus.cc:504] T 00000000000000000000000000000000 P a5810eb7ebb54e919ddee57060134845 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5810eb7ebb54e919ddee57060134845" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 42481 } }
I20260806 11:41:11.326314 12550 leader_election.cc:302] T 00000000000000000000000000000000 P a5810eb7ebb54e919ddee57060134845 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a5810eb7ebb54e919ddee57060134845; no voters:
I20260806 11:41:11.326678 12550 leader_election.cc:288] T 00000000000000000000000000000000 P a5810eb7ebb54e919ddee57060134845 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 11:41:11.326812 12559 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a5810eb7ebb54e919ddee57060134845 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:41:11.327082 12559 raft_consensus.cc:686] T 00000000000000000000000000000000 P a5810eb7ebb54e919ddee57060134845 [term 1 LEADER]: Becoming Leader. State: Replica: a5810eb7ebb54e919ddee57060134845, State: Running, Role: LEADER
I20260806 11:41:11.327668 12559 consensus_queue.cc:237] T 00000000000000000000000000000000 P a5810eb7ebb54e919ddee57060134845 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5810eb7ebb54e919ddee57060134845" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 42481 } }
I20260806 11:41:11.328125 12550 sys_catalog.cc:564] T 00000000000000000000000000000000 P a5810eb7ebb54e919ddee57060134845 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 11:41:11.331420 12561 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5810eb7ebb54e919ddee57060134845 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a5810eb7ebb54e919ddee57060134845. Latest consensus state: current_term: 1 leader_uuid: "a5810eb7ebb54e919ddee57060134845" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5810eb7ebb54e919ddee57060134845" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 42481 } } }
I20260806 11:41:11.331583 12561 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5810eb7ebb54e919ddee57060134845 [sys.catalog]: This master's current role is: LEADER
I20260806 11:41:11.331974 12560 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5810eb7ebb54e919ddee57060134845 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a5810eb7ebb54e919ddee57060134845" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5810eb7ebb54e919ddee57060134845" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 42481 } } }
I20260806 11:41:11.332114 12560 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5810eb7ebb54e919ddee57060134845 [sys.catalog]: This master's current role is: LEADER
W20260806 11:41:11.334295 12575 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a5810eb7ebb54e919ddee57060134845: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 11:41:11.334394 12575 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 11:41:11.334524 12578 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 11:41:11.335546 12578 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 11:41:11.336861 12553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:11.327341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36821 (local address 127.21.14.254:42481)
0806 11:41:11.327483 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:11.327489 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:11.327810 (+ 321us) server_negotiation.cc:407] Connection header received
0806 11:41:11.327940 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:11.327943 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:11.327993 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:11.328080 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:11.329098 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:11.329974 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:11.331179 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:11.331516 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:11.331635 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:41:11.333413 (+ 1778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:41:11.333417 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:41:11.333419 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:41:11.333447 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:41:11.335103 (+ 1656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:41:11.335534 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:41:11.335538 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:41:11.335540 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:41:11.335655 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:41:11.335938 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:41:11.335942 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:41:11.335943 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:41:11.336143 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:41:11.336288 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 11:41:11.336542 (+ 254us) server_negotiation.cc:299] Negotiation successful
0806 11:41:11.336677 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 11:41:11.342463 12578 catalog_manager.cc:1347] Generated new cluster ID: 82561c605f784e4fa33058dac3f3a2c8
I20260806 11:41:11.342545 12578 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 11:41:11.354321 12578 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 11:41:11.355749 12578 catalog_manager.cc:1504] Loading token signing keys...
I20260806 11:41:11.365335 12578 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a5810eb7ebb54e919ddee57060134845: Generated new TSK 0
I20260806 11:41:11.366271 12578 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 11:41:11.371224 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 12478
2026-08-06T11:41:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1497 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786016169045816-21563-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 11:41:11 dist-test-slave-7c35 krb5kdc[12583](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 11:41:11 dist-test-slave-7c35 krb5kdc[12583](info): set up 2 sockets
Aug 06 11:41:11 dist-test-slave-7c35 krb5kdc[12583](info): commencing operation
krb5kdc: starting...
W20260806 11:41:12.477725 21563 mini_kdc.cc:121] Time spent starting KDC: real 1.084s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Aug 06 11:41:12 dist-test-slave-7c35 krb5kdc[12583](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016472, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T11:41:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T11:41:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.14.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab.
Entry for principal kudu/127.21.14.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab.
I20260806 11:41:12.631086 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786016169045816-21563-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786016169045816-21563-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786016169045816-21563-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:38521
--webserver_interface=127.21.14.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab
--principal=kudu/127.21.14.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:38059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.14.254:38521
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:41:12.752135 12596 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:41:12.752511 12596 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:41:12.752590 12596 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:41:12.756927 12596 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 11:41:12.757051 12596 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:41:12.757098 12596 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:41:12.757156 12596 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 11:41:12.757195 12596 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 11:41:12.762928 12596 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:38059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786016169045816-21563-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786016169045816-21563-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.14.254:38521
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab
--principal=kudu/127.21.14.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:38521
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.14.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12596
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786016169045816-21563-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:41:12.764202 12596 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:41:12.765358 12596 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:41:12.774804 12601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:41:12.774961 12596 server_base.cc:1034] running on GCE node
W20260806 11:41:12.774804 12602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:41:12.774837 12605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:41:12.775930 12596 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:41:12.777120 12596 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:41:12.778347 12596 hybrid_clock.cc:648] HybridClock initialized: now 1786016472778308 us; error 53 us; skew 500 ppm
Aug 06 11:41:12 dist-test-slave-7c35 krb5kdc[12583](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016472, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:41:12.781617 12596 init.cc:377] Logged in from keytab as kudu/127.21.14.254@KRBTEST.COM (short username kudu)
I20260806 11:41:12.782750 12596 webserver.cc:460] Webserver started at http://127.21.14.254:36881/ using document root <none> and password file <none>
I20260806 11:41:12.783433 12596 fs_manager.cc:359] Metadata directory not provided
I20260806 11:41:12.783557 12596 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:41:12.783828 12596 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:41:12.786360 12596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786016169045816-21563-0/minicluster-data/master-0/data/instance:
uuid: "ecadda18310f4c5a82543cf32ec0d8a3"
format_stamp: "Formatted at 2026-08-06 11:41:12 on dist-test-slave-7c35"
server_key: "7d7f86ee4c7738a736b0ca1d29e7d232"
server_key_iv: "30eca26503671e26d209e64dbb6e9cb7"
server_key_version: "encryptionkey@0"
I20260806 11:41:12.787133 12596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786016169045816-21563-0/minicluster-data/master-0/wal/instance:
uuid: "ecadda18310f4c5a82543cf32ec0d8a3"
format_stamp: "Formatted at 2026-08-06 11:41:12 on dist-test-slave-7c35"
server_key: "7d7f86ee4c7738a736b0ca1d29e7d232"
server_key_iv: "30eca26503671e26d209e64dbb6e9cb7"
server_key_version: "encryptionkey@0"
I20260806 11:41:12.791603 12596 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 11:41:12.794456 12612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:41:12.795915 12596 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 11:41:12.796088 12596 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786016169045816-21563-0/minicluster-data/master-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786016169045816-21563-0/minicluster-data/master-0/wal
uuid: "ecadda18310f4c5a82543cf32ec0d8a3"
format_stamp: "Formatted at 2026-08-06 11:41:12 on dist-test-slave-7c35"
server_key: "7d7f86ee4c7738a736b0ca1d29e7d232"
server_key_iv: "30eca26503671e26d209e64dbb6e9cb7"
server_key_version: "encryptionkey@0"
I20260806 11:41:12.796231 12596 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786016169045816-21563-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786016169045816-21563-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786016169045816-21563-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:41:12.815804 12596 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:41:12.819554 12596 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:41:12.819825 12596 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:41:12.828589 12596 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.254:38521
I20260806 11:41:12.828619 12666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.254:38521 every 8 connection(s)
I20260806 11:41:12.829742 12596 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
I20260806 11:41:12.832864 12667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:41:12.839105 12667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecadda18310f4c5a82543cf32ec0d8a3: Bootstrap starting.
I20260806 11:41:12.839574 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 12596
I20260806 11:41:12.839722 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786016169045816-21563-0/minicluster-data/master-0/wal/instance
I20260806 11:41:12.840029 21563 external_mini_cluster.cc:1428] Setting key 5755acc4665d128d1c9ae03703cdf818
I20260806 11:41:12.842207 12667 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ecadda18310f4c5a82543cf32ec0d8a3: Neither blocks nor log segments found. Creating new log.
I20260806 11:41:12.843109 12667 log.cc:824] T 00000000000000000000000000000000 P ecadda18310f4c5a82543cf32ec0d8a3: Log is configured to *not* fsync() on all Append() calls
I20260806 11:41:12.845148 12667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecadda18310f4c5a82543cf32ec0d8a3: No bootstrap required, opened a new log
I20260806 11:41:12.849116 12667 raft_consensus.cc:348] T 00000000000000000000000000000000 P ecadda18310f4c5a82543cf32ec0d8a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecadda18310f4c5a82543cf32ec0d8a3" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 38521 } }
I20260806 11:41:12.849399 12667 raft_consensus.cc:374] T 00000000000000000000000000000000 P ecadda18310f4c5a82543cf32ec0d8a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:41:12.849504 12667 raft_consensus.cc:729] T 00000000000000000000000000000000 P ecadda18310f4c5a82543cf32ec0d8a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecadda18310f4c5a82543cf32ec0d8a3, State: Initialized, Role: FOLLOWER
I20260806 11:41:12.849997 12667 consensus_queue.cc:260] T 00000000000000000000000000000000 P ecadda18310f4c5a82543cf32ec0d8a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecadda18310f4c5a82543cf32ec0d8a3" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 38521 } }
I20260806 11:41:12.850190 12667 raft_consensus.cc:388] T 00000000000000000000000000000000 P ecadda18310f4c5a82543cf32ec0d8a3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 11:41:12.850282 12667 raft_consensus.cc:482] T 00000000000000000000000000000000 P ecadda18310f4c5a82543cf32ec0d8a3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 11:41:12.850407 12667 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ecadda18310f4c5a82543cf32ec0d8a3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:41:12.850742 12671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:12.841624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48037 (local address 127.21.14.254:38521)
0806 11:41:12.842114 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:12.842132 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:12.842172 (+ 40us) server_negotiation.cc:407] Connection header received
0806 11:41:12.843037 (+ 865us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:12.843058 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:12.843508 (+ 450us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:12.843888 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:12.844359 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:12.845577 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:12.846639 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:12.847241 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:12.847449 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:41:12.848462 (+ 1013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:41:12.848468 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:41:12.848483 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 11:41:12.848944 (+ 461us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:41:12.849006 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 11:41:12.849056 (+ 50us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:41:12.849187 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 11:41:12.849358 (+ 171us) server_negotiation.cc:299] Negotiation successful
0806 11:41:12.849626 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":139,"threads_started":1}
I20260806 11:41:12.851702 12667 raft_consensus.cc:504] T 00000000000000000000000000000000 P ecadda18310f4c5a82543cf32ec0d8a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecadda18310f4c5a82543cf32ec0d8a3" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 38521 } }
I20260806 11:41:12.852079 12667 leader_election.cc:302] T 00000000000000000000000000000000 P ecadda18310f4c5a82543cf32ec0d8a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ecadda18310f4c5a82543cf32ec0d8a3; no voters:
I20260806 11:41:12.852428 12667 leader_election.cc:288] T 00000000000000000000000000000000 P ecadda18310f4c5a82543cf32ec0d8a3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 11:41:12.852597 12673 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ecadda18310f4c5a82543cf32ec0d8a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:41:12.852845 12673 raft_consensus.cc:686] T 00000000000000000000000000000000 P ecadda18310f4c5a82543cf32ec0d8a3 [term 1 LEADER]: Becoming Leader. State: Replica: ecadda18310f4c5a82543cf32ec0d8a3, State: Running, Role: LEADER
I20260806 11:41:12.853229 12673 consensus_queue.cc:237] T 00000000000000000000000000000000 P ecadda18310f4c5a82543cf32ec0d8a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecadda18310f4c5a82543cf32ec0d8a3" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 38521 } }
I20260806 11:41:12.853875 12667 sys_catalog.cc:564] T 00000000000000000000000000000000 P ecadda18310f4c5a82543cf32ec0d8a3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 11:41:12.854553 21563 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 11:41:12.855283 12674 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecadda18310f4c5a82543cf32ec0d8a3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ecadda18310f4c5a82543cf32ec0d8a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecadda18310f4c5a82543cf32ec0d8a3" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 38521 } } }
I20260806 11:41:12.855413 12674 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecadda18310f4c5a82543cf32ec0d8a3 [sys.catalog]: This master's current role is: LEADER
I20260806 11:41:12.855754 12675 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecadda18310f4c5a82543cf32ec0d8a3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ecadda18310f4c5a82543cf32ec0d8a3. Latest consensus state: current_term: 1 leader_uuid: "ecadda18310f4c5a82543cf32ec0d8a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecadda18310f4c5a82543cf32ec0d8a3" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 38521 } } }
I20260806 11:41:12.855860 12675 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecadda18310f4c5a82543cf32ec0d8a3 [sys.catalog]: This master's current role is: LEADER
I20260806 11:41:12.855866 12682 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 11:41:12.859565 12682 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 11:41:12.862969 12671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:12.857061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58729 (local address 127.21.14.254:38521)
0806 11:41:12.857182 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:12.857187 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:12.857766 (+ 579us) server_negotiation.cc:407] Connection header received
0806 11:41:12.857977 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:12.857980 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:12.858047 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:12.858105 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:12.858594 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:12.859462 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:12.860986 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:12.861352 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:12.861470 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:41:12.862443 (+ 973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:41:12.862448 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:41:12.862453 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 11:41:12.862472 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:41:12.862501 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 11:41:12.862513 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:41:12.862617 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 11:41:12.862720 (+ 103us) server_negotiation.cc:299] Negotiation successful
0806 11:41:12.862779 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 11:41:12.866463 12682 catalog_manager.cc:1347] Generated new cluster ID: 6016582bc06b40449148ed86fdece4f7
I20260806 11:41:12.866560 12682 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 11:41:12.889868 12682 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 11:41:12.891309 12682 catalog_manager.cc:1504] Loading token signing keys...
I20260806 11:41:12.896615 12682 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ecadda18310f4c5a82543cf32ec0d8a3: Generated new TSK 0
I20260806 11:41:12.897516 12682 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 11:41:12.909925 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 12596
2026-08-06T11:41:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1539 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786016169045816-21563-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 11:41:12 dist-test-slave-7c35 krb5kdc[12700](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 11:41:12 dist-test-slave-7c35 krb5kdc[12700](info): set up 2 sockets
Aug 06 11:41:12 dist-test-slave-7c35 krb5kdc[12700](info): commencing operation
krb5kdc: starting...
W20260806 11:41:13.994323 21563 mini_kdc.cc:121] Time spent starting KDC: real 1.061s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Aug 06 11:41:14 dist-test-slave-7c35 krb5kdc[12700](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016474, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T11:41:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T11:41:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.14.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab.
Entry for principal kudu/127.21.14.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab.
I20260806 11:41:14.139925 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786016169045816-21563-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786016169045816-21563-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786016169045816-21563-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:40517
--webserver_interface=127.21.14.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab
--principal=kudu/127.21.14.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:45523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.14.254:40517
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:41:14.263267 12714 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:41:14.263598 12714 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:41:14.263645 12714 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:41:14.267905 12714 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 11:41:14.267973 12714 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:41:14.267994 12714 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:41:14.268011 12714 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 11:41:14.268028 12714 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 11:41:14.273618 12714 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:45523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786016169045816-21563-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786016169045816-21563-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.14.254:40517
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab
--principal=kudu/127.21.14.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:40517
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.14.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12714
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786016169045816-21563-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:41:14.274768 12714 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:41:14.275893 12714 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:41:14.284835 12722 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:41:14.284888 12714 server_base.cc:1034] running on GCE node
W20260806 11:41:14.284825 12720 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:41:14.284821 12719 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:41:14.285809 12714 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:41:14.287041 12714 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:41:14.288262 12714 hybrid_clock.cc:648] HybridClock initialized: now 1786016474288233 us; error 35 us; skew 500 ppm
Aug 06 11:41:14 dist-test-slave-7c35 krb5kdc[12700](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016474, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:41:14.291297 12714 init.cc:377] Logged in from keytab as kudu/127.21.14.254@KRBTEST.COM (short username kudu)
I20260806 11:41:14.292294 12714 webserver.cc:460] Webserver started at http://127.21.14.254:36479/ using document root <none> and password file <none>
I20260806 11:41:14.292881 12714 fs_manager.cc:359] Metadata directory not provided
I20260806 11:41:14.292944 12714 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:41:14.293169 12714 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:41:14.295373 12714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786016169045816-21563-0/minicluster-data/master-0/data/instance:
uuid: "a4e8bffda07d46f28d0420dd762ba42b"
format_stamp: "Formatted at 2026-08-06 11:41:14 on dist-test-slave-7c35"
server_key: "349d9d4ec347c43402fda406f55151dc"
server_key_iv: "fd15a7959b5eb9c77b2a4b768f289c8d"
server_key_version: "encryptionkey@0"
I20260806 11:41:14.295957 12714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786016169045816-21563-0/minicluster-data/master-0/wal/instance:
uuid: "a4e8bffda07d46f28d0420dd762ba42b"
format_stamp: "Formatted at 2026-08-06 11:41:14 on dist-test-slave-7c35"
server_key: "349d9d4ec347c43402fda406f55151dc"
server_key_iv: "fd15a7959b5eb9c77b2a4b768f289c8d"
server_key_version: "encryptionkey@0"
I20260806 11:41:14.300464 12714 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 11:41:14.303133 12729 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:41:14.304508 12714 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 11:41:14.304656 12714 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786016169045816-21563-0/minicluster-data/master-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786016169045816-21563-0/minicluster-data/master-0/wal
uuid: "a4e8bffda07d46f28d0420dd762ba42b"
format_stamp: "Formatted at 2026-08-06 11:41:14 on dist-test-slave-7c35"
server_key: "349d9d4ec347c43402fda406f55151dc"
server_key_iv: "fd15a7959b5eb9c77b2a4b768f289c8d"
server_key_version: "encryptionkey@0"
I20260806 11:41:14.304780 12714 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786016169045816-21563-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786016169045816-21563-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786016169045816-21563-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:41:14.309756 12714 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:41:14.313110 12714 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:41:14.313330 12714 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:41:14.321339 12714 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.254:40517
I20260806 11:41:14.321365 12781 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.254:40517 every 8 connection(s)
I20260806 11:41:14.322378 12714 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
I20260806 11:41:14.325572 12782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:41:14.328429 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 12714
I20260806 11:41:14.328594 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786016169045816-21563-0/minicluster-data/master-0/wal/instance
I20260806 11:41:14.328922 21563 external_mini_cluster.cc:1428] Setting key 1eb7b764e96dee1e28d78e2cdf7b7bf6
I20260806 11:41:14.332911 12782 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4e8bffda07d46f28d0420dd762ba42b: Bootstrap starting.
I20260806 11:41:14.334771 12785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:14.330629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49211 (local address 127.21.14.254:40517)
0806 11:41:14.331186 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:14.331203 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:14.331244 (+ 41us) server_negotiation.cc:407] Connection header received
0806 11:41:14.331929 (+ 685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:14.331947 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:14.332282 (+ 335us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:14.332645 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:14.333545 (+ 900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:41:14.333557 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:41:14.333571 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 11:41:14.333582 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:41:14.333636 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 11:41:14.333657 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:41:14.333814 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 11:41:14.333889 (+ 75us) server_negotiation.cc:299] Negotiation successful
0806 11:41:14.334053 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":201,"threads_started":1}
I20260806 11:41:14.335659 12782 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a4e8bffda07d46f28d0420dd762ba42b: Neither blocks nor log segments found. Creating new log.
I20260806 11:41:14.336496 12782 log.cc:824] T 00000000000000000000000000000000 P a4e8bffda07d46f28d0420dd762ba42b: Log is configured to *not* fsync() on all Append() calls
I20260806 11:41:14.337785 21563 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 11:41:14.338932 12782 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4e8bffda07d46f28d0420dd762ba42b: No bootstrap required, opened a new log
I20260806 11:41:14.341679 12785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:14.339937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34709 (local address 127.21.14.254:40517)
0806 11:41:14.340054 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:14.340060 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:14.340276 (+ 216us) server_negotiation.cc:407] Connection header received
0806 11:41:14.340396 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:14.340398 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:14.340427 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:14.340491 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:14.341257 (+ 766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:41:14.341260 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:41:14.341262 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 11:41:14.341272 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:41:14.341296 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 11:41:14.341307 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:41:14.341377 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 11:41:14.341494 (+ 117us) server_negotiation.cc:299] Negotiation successful
0806 11:41:14.341525 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 11:41:14.343977 12782 raft_consensus.cc:348] T 00000000000000000000000000000000 P a4e8bffda07d46f28d0420dd762ba42b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4e8bffda07d46f28d0420dd762ba42b" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 40517 } }
I20260806 11:41:14.344267 12782 raft_consensus.cc:374] T 00000000000000000000000000000000 P a4e8bffda07d46f28d0420dd762ba42b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 11:41:14.344334 12782 raft_consensus.cc:729] T 00000000000000000000000000000000 P a4e8bffda07d46f28d0420dd762ba42b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4e8bffda07d46f28d0420dd762ba42b, State: Initialized, Role: FOLLOWER
I20260806 11:41:14.344798 12782 consensus_queue.cc:260] T 00000000000000000000000000000000 P a4e8bffda07d46f28d0420dd762ba42b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4e8bffda07d46f28d0420dd762ba42b" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 40517 } }
I20260806 11:41:14.344967 12782 raft_consensus.cc:388] T 00000000000000000000000000000000 P a4e8bffda07d46f28d0420dd762ba42b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 11:41:14.345068 12782 raft_consensus.cc:482] T 00000000000000000000000000000000 P a4e8bffda07d46f28d0420dd762ba42b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 11:41:14.345182 12782 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a4e8bffda07d46f28d0420dd762ba42b [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:41:14.346333 12782 raft_consensus.cc:504] T 00000000000000000000000000000000 P a4e8bffda07d46f28d0420dd762ba42b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4e8bffda07d46f28d0420dd762ba42b" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 40517 } }
I20260806 11:41:14.346737 12782 leader_election.cc:302] T 00000000000000000000000000000000 P a4e8bffda07d46f28d0420dd762ba42b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a4e8bffda07d46f28d0420dd762ba42b; no voters:
I20260806 11:41:14.347177 12782 leader_election.cc:288] T 00000000000000000000000000000000 P a4e8bffda07d46f28d0420dd762ba42b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 11:41:14.347296 12792 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a4e8bffda07d46f28d0420dd762ba42b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:41:14.347563 12792 raft_consensus.cc:686] T 00000000000000000000000000000000 P a4e8bffda07d46f28d0420dd762ba42b [term 1 LEADER]: Becoming Leader. State: Replica: a4e8bffda07d46f28d0420dd762ba42b, State: Running, Role: LEADER
I20260806 11:41:14.347904 12792 consensus_queue.cc:237] T 00000000000000000000000000000000 P a4e8bffda07d46f28d0420dd762ba42b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4e8bffda07d46f28d0420dd762ba42b" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 40517 } }
I20260806 11:41:14.348732 12782 sys_catalog.cc:564] T 00000000000000000000000000000000 P a4e8bffda07d46f28d0420dd762ba42b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 11:41:14.349149 12794 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4e8bffda07d46f28d0420dd762ba42b [sys.catalog]: SysCatalogTable state changed. Reason: New leader a4e8bffda07d46f28d0420dd762ba42b. Latest consensus state: current_term: 1 leader_uuid: "a4e8bffda07d46f28d0420dd762ba42b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4e8bffda07d46f28d0420dd762ba42b" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 40517 } } }
I20260806 11:41:14.349285 12794 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4e8bffda07d46f28d0420dd762ba42b [sys.catalog]: This master's current role is: LEADER
I20260806 11:41:14.349696 12797 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 11:41:14.353219 12797 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 11:41:14.349625 12793 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4e8bffda07d46f28d0420dd762ba42b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a4e8bffda07d46f28d0420dd762ba42b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4e8bffda07d46f28d0420dd762ba42b" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 40517 } } }
I20260806 11:41:14.355126 12793 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4e8bffda07d46f28d0420dd762ba42b [sys.catalog]: This master's current role is: LEADER
I20260806 11:41:14.359344 12797 catalog_manager.cc:1347] Generated new cluster ID: 9e0aa26a7a2a41e3aca1db4e8716fc0d
I20260806 11:41:14.359427 12797 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 11:41:14.374974 12797 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 11:41:14.375336 12797 catalog_manager.cc:1504] Loading token signing keys...
I20260806 11:41:14.384083 12797 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a4e8bffda07d46f28d0420dd762ba42b: Generated new TSK 0
I20260806 11:41:14.384847 12797 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 11:41:14.390771 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 12714
2026-08-06T11:41:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1481 ms)
[----------] 18 tests from AuthTokenIssuingTest (27253 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 11:41:14 dist-test-slave-7c35 krb5kdc[12815](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 11:41:14 dist-test-slave-7c35 krb5kdc[12815](info): set up 2 sockets
Aug 06 11:41:14 dist-test-slave-7c35 krb5kdc[12815](info): commencing operation
krb5kdc: starting...
W20260806 11:41:15.471138 21563 mini_kdc.cc:121] Time spent starting KDC: real 1.057s user 0.005s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Aug 06 11:41:15 dist-test-slave-7c35 krb5kdc[12815](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016475, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T11:41:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T11:41:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.14.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab.
Entry for principal kudu/127.21.14.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab.
I20260806 11:41:15.611008 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:35671
--webserver_interface=127.21.14.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab
--principal=kudu/127.21.14.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:33535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.14.254:35671,127.21.14.253:34937,127.21.14.252:33185
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:41:15.734817 12828 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 11:41:15.735208 12828 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:41:15.735262 12828 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:41:15.735337 12828 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:41:15.739485 12828 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 11:41:15.739575 12828 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:41:15.739612 12828 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:41:15.739651 12828 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 11:41:15.739686 12828 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 11:41:15.745155 12828 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:33535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.14.254:35671,127.21.14.253:34937,127.21.14.252:33185
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab
--principal=kudu/127.21.14.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:35671
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.14.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12828
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:41:15.746332 12828 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:41:15.747534 12828 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:41:15.756564 12834 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:41:15.756565 12833 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:41:15.756570 12836 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:41:15.756606 12828 server_base.cc:1034] running on GCE node
I20260806 11:41:15.757464 12828 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:41:15.758700 12828 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:41:15.759900 12828 hybrid_clock.cc:648] HybridClock initialized: now 1786016475759885 us; error 31 us; skew 500 ppm
Aug 06 11:41:15 dist-test-slave-7c35 krb5kdc[12815](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016475, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:41:15.767196 12828 init.cc:377] Logged in from keytab as kudu/127.21.14.254@KRBTEST.COM (short username kudu)
I20260806 11:41:15.768221 12828 webserver.cc:460] Webserver started at http://127.21.14.254:38219/ using document root <none> and password file <none>
I20260806 11:41:15.768836 12828 fs_manager.cc:359] Metadata directory not provided
I20260806 11:41:15.768934 12828 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:41:15.769194 12828 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:41:15.771489 12828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/minicluster-data/master-0/data/instance:
uuid: "b300ea3e113d4528911b80e89f04bfdd"
format_stamp: "Formatted at 2026-08-06 11:41:15 on dist-test-slave-7c35"
server_key: "12973a96cdcab7f02414f88a44dbf42f"
server_key_iv: "36c14493455e535edf3931ff8011179c"
server_key_version: "encryptionkey@0"
I20260806 11:41:15.772104 12828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/minicluster-data/master-0/wal/instance:
uuid: "b300ea3e113d4528911b80e89f04bfdd"
format_stamp: "Formatted at 2026-08-06 11:41:15 on dist-test-slave-7c35"
server_key: "12973a96cdcab7f02414f88a44dbf42f"
server_key_iv: "36c14493455e535edf3931ff8011179c"
server_key_version: "encryptionkey@0"
I20260806 11:41:15.776145 12828 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 11:41:15.778805 12843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:41:15.780176 12828 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 11:41:15.780332 12828 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/minicluster-data/master-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/minicluster-data/master-0/wal
uuid: "b300ea3e113d4528911b80e89f04bfdd"
format_stamp: "Formatted at 2026-08-06 11:41:15 on dist-test-slave-7c35"
server_key: "12973a96cdcab7f02414f88a44dbf42f"
server_key_iv: "36c14493455e535edf3931ff8011179c"
server_key_version: "encryptionkey@0"
I20260806 11:41:15.780462 12828 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:41:15.799733 12828 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:41:15.802995 12828 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:41:15.803233 12828 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:41:15.811789 12828 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.254:35671
I20260806 11:41:15.812124 12897 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.254:35671 every 8 connection(s)
I20260806 11:41:15.812875 12828 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
I20260806 11:41:15.816113 12898 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:41:15.819729 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 12828
I20260806 11:41:15.819921 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/minicluster-data/master-0/wal/instance
I20260806 11:41:15.820256 21563 external_mini_cluster.cc:1428] Setting key 38bd10bce7e09dda0e3ed2a06ef1de05
I20260806 11:41:15.820871 12898 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.14.254" port: 35671 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.21.14.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.253@KRBTEST.COM" created.
Aug 06 11:41:15 dist-test-slave-7c35 krb5kdc[12815](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016475, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.254@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:41:15.839907 12901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:15.823983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37866 (local address 127.21.14.254:35671)
0806 11:41:15.824478 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:15.824499 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:15.824794 (+ 295us) server_negotiation.cc:407] Connection header received
0806 11:41:15.825928 (+ 1134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:15.825933 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:15.826132 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:15.826327 (+ 195us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:15.827299 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:15.828221 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:15.829244 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:15.829680 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:15.829867 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:41:15.833919 (+ 4052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:41:15.833931 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:41:15.833951 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:41:15.833987 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:41:15.836139 (+ 2152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:41:15.836725 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:41:15.836731 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:41:15.836732 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:41:15.836852 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:41:15.837310 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:41:15.837313 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:41:15.837314 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:41:15.837889 (+ 575us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:41:15.838052 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0806 11:41:15.838608 (+ 556us) server_negotiation.cc:299] Negotiation successful
0806 11:41:15.839004 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":36,"server-negotiator.queue_time_us":329,"thread_start_us":161,"threads_started":1}
I20260806 11:41:15.839912 12902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:15.823984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:35671 (local address 127.0.0.1:37866)
0806 11:41:15.824638 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0806 11:41:15.824678 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 11:41:15.825598 (+ 920us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:41:15.826325 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:41:15.826331 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:41:15.826821 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:41:15.827142 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:41:15.827157 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:41:15.828357 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:41:15.828360 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:41:15.829121 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:41:15.829130 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:41:15.829852 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:41:15.829856 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:41:15.829976 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:41:15.830667 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:41:15.830686 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:41:15.833700 (+ 3014us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:41:15.836287 (+ 2587us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:41:15.836300 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:41:15.836313 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:41:15.836605 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:41:15.836961 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:41:15.836967 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:41:15.836972 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:41:15.837156 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:41:15.838025 (+ 869us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:41:15.838037 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:41:15.838240 (+ 203us) client_negotiation.cc:769] Sending connection context
0806 11:41:15.838502 (+ 262us) client_negotiation.cc:240] Negotiation successful
0806 11:41:15.839119 (+ 617us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"mutex_wait_us":192,"spinlock_wait_cycles":768,"thread_start_us":197,"threads_started":1}
I20260806 11:41:15.842659 12898 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.14.253" port: 34937 } has no permanent_uuid. Determining permanent_uuid...
I20260806 11:41:15.843495 12902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:15.843229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.253:34937 (local address 127.0.0.1:45534)
0806 11:41:15.843356 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 11:41:15.843431 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.253:34937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260806 11:41:15.843742 12846 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.14.253:34937: connect: Connection refused (error 111)
I20260806 11:41:15.844727 12902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:15.844489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.253:34937 (local address 127.0.0.1:45542)
0806 11:41:15.844627 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 11:41:15.844673 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.253:34937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260806 11:41:15.844914 12898 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.14.253:34937: Network error: Client connection negotiation failed: client connection to 127.21.14.253:34937: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.253.keytab.
Entry for principal kudu/127.21.14.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.253.keytab.
I20260806 11:41:15.855026 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.253:34937
--webserver_interface=127.21.14.253
--webserver_port=0
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.253.keytab
--principal=kudu/127.21.14.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:33535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.14.254:35671,127.21.14.253:34937,127.21.14.252:33185
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 11:41:15.894164 12898 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.14.253" port: 34937 } attempt: 1
I20260806 11:41:15.895218 12902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:15.894843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.253:34937 (local address 127.0.0.1:45546)
0806 11:41:15.895065 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0806 11:41:15.895135 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.253:34937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260806 11:41:15.896049 12902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:15.895832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.253:34937 (local address 127.0.0.1:45556)
0806 11:41:15.895938 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0806 11:41:15.895990 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.253:34937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260806 11:41:15.896210 12898 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.14.253:34937: Network error: Client connection negotiation failed: client connection to 127.21.14.253:34937: connect: Connection refused (error 111)
I20260806 11:41:15.964416 12898 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.14.253" port: 34937 } attempt: 2
I20260806 11:41:15.965359 12902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:15.965041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.253:34937 (local address 127.0.0.1:45564)
0806 11:41:15.965209 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 11:41:15.965277 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.253:34937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260806 11:41:15.966123 12902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:15.965918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.253:34937 (local address 127.0.0.1:45580)
0806 11:41:15.966016 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0806 11:41:15.966065 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.253:34937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260806 11:41:15.966271 12898 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.14.253:34937: Network error: Client connection negotiation failed: client connection to 127.21.14.253:34937: connect: Connection refused (error 111)
W20260806 11:41:15.975746 12905 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 11:41:15.976063 12905 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:41:15.976116 12905 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:41:15.976192 12905 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:41:15.980429 12905 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 11:41:15.980502 12905 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:41:15.980522 12905 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:41:15.980540 12905 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 11:41:15.980556 12905 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 11:41:15.986413 12905 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:33535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.21.14.254:35671,127.21.14.253:34937,127.21.14.252:33185
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.253.keytab
--principal=kudu/127.21.14.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.253:34937
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.21.14.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12905
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:41:15.987677 12905 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:41:15.988803 12905 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:41:15.998157 12911 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:41:15.998174 12910 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:41:15.998198 12905 server_base.cc:1034] running on GCE node
W20260806 11:41:15.998483 12913 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:41:15.999231 12905 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:41:16.000474 12905 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:41:16.001648 12905 hybrid_clock.cc:648] HybridClock initialized: now 1786016476001632 us; error 29 us; skew 500 ppm
Aug 06 11:41:16 dist-test-slave-7c35 krb5kdc[12815](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016476, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:41:16.004650 12905 init.cc:377] Logged in from keytab as kudu/127.21.14.253@KRBTEST.COM (short username kudu)
I20260806 11:41:16.005592 12905 webserver.cc:460] Webserver started at http://127.21.14.253:41559/ using document root <none> and password file <none>
I20260806 11:41:16.006155 12905 fs_manager.cc:359] Metadata directory not provided
I20260806 11:41:16.006217 12905 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:41:16.006397 12905 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:41:16.008625 12905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/minicluster-data/master-1/data/instance:
uuid: "3e6a50bf43cd4cbcad84c8975b5542a3"
format_stamp: "Formatted at 2026-08-06 11:41:16 on dist-test-slave-7c35"
server_key: "ed0dc21547f96fe87c39b06659e289e9"
server_key_iv: "3762def903760913ddb6c49c58a693cf"
server_key_version: "encryptionkey@0"
I20260806 11:41:16.009181 12905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/minicluster-data/master-1/wal/instance:
uuid: "3e6a50bf43cd4cbcad84c8975b5542a3"
format_stamp: "Formatted at 2026-08-06 11:41:16 on dist-test-slave-7c35"
server_key: "ed0dc21547f96fe87c39b06659e289e9"
server_key_iv: "3762def903760913ddb6c49c58a693cf"
server_key_version: "encryptionkey@0"
I20260806 11:41:16.013350 12905 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 11:41:16.015988 12920 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:41:16.017331 12905 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 11:41:16.017482 12905 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/minicluster-data/master-1/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/minicluster-data/master-1/wal
uuid: "3e6a50bf43cd4cbcad84c8975b5542a3"
format_stamp: "Formatted at 2026-08-06 11:41:16 on dist-test-slave-7c35"
server_key: "ed0dc21547f96fe87c39b06659e289e9"
server_key_iv: "3762def903760913ddb6c49c58a693cf"
server_key_version: "encryptionkey@0"
I20260806 11:41:16.017611 12905 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:41:16.032140 12905 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:41:16.035440 12905 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:41:16.035669 12905 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:41:16.043512 12905 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.253:34937
I20260806 11:41:16.043816 12975 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.253:34937 every 8 connection(s)
I20260806 11:41:16.044606 12905 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/minicluster-data/master-1/data/info.pb
I20260806 11:41:16.047645 12976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:41:16.051687 12976 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.14.254" port: 35671 } has no permanent_uuid. Determining permanent_uuid...
I20260806 11:41:16.053153 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 12905
I20260806 11:41:16.053303 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/minicluster-data/master-1/wal/instance
I20260806 11:41:16.053593 21563 external_mini_cluster.cc:1428] Setting key c727e83f6dd345c256139a4c73c8a3c3
I20260806 11:41:16.057464 12898 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.14.253" port: 34937 } attempt: 3
Aug 06 11:41:16 dist-test-slave-7c35 krb5kdc[12815](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016476, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.253@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.14.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.252@KRBTEST.COM" created.
Aug 06 11:41:16 dist-test-slave-7c35 krb5kdc[12815](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016475, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.254@KRBTEST.COM for kudu/127.21.14.253@KRBTEST.COM
I20260806 11:41:16.068897 12901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:16.053755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37882 (local address 127.21.14.254:35671)
0806 11:41:16.053949 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:16.053955 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:16.054878 (+ 923us) server_negotiation.cc:407] Connection header received
0806 11:41:16.055905 (+ 1027us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:16.055908 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:16.055975 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:16.056060 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:16.057268 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:16.058015 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:16.059822 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:16.060273 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:16.060409 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:41:16.063491 (+ 3082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:41:16.063498 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:41:16.063504 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:41:16.063535 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:41:16.066007 (+ 2472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:41:16.066667 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:41:16.066673 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:41:16.066675 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:41:16.066772 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:41:16.067253 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:41:16.067256 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:41:16.067257 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:41:16.067662 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:41:16.067792 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 11:41:16.068556 (+ 764us) server_negotiation.cc:299] Negotiation successful
0806 11:41:16.068722 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 11:41:16.069988 12978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:16.054176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:35671 (local address 127.0.0.1:37882)
0806 11:41:16.054692 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0806 11:41:16.054750 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 11:41:16.055677 (+ 927us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:41:16.056235 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:41:16.056243 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:41:16.056604 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:41:16.057110 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:41:16.057125 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:41:16.058384 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:41:16.058388 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:41:16.059692 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:41:16.059702 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:41:16.060449 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:41:16.060455 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:41:16.060699 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:41:16.061332 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:41:16.061349 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:41:16.063298 (+ 1949us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:41:16.066152 (+ 2854us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:41:16.066168 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:41:16.066185 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:41:16.066486 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:41:16.066916 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:41:16.066920 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:41:16.066922 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:41:16.067108 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:41:16.067863 (+ 755us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:41:16.067871 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:41:16.068259 (+ 388us) client_negotiation.cc:769] Sending connection context
0806 11:41:16.068474 (+ 215us) client_negotiation.cc:240] Negotiation successful
0806 11:41:16.068910 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"mutex_wait_us":5,"thread_start_us":153,"threads_started":1}
I20260806 11:41:16.071931 12976 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.14.253" port: 34937 } has no permanent_uuid. Determining permanent_uuid...
I20260806 11:41:16.073393 12902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:16.057957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.253:34937 (local address 127.0.0.1:45582)
0806 11:41:16.058183 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0806 11:41:16.058194 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 11:41:16.058246 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:41:16.059067 (+ 821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:41:16.059072 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:41:16.059294 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:41:16.059473 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:41:16.059483 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:41:16.060582 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:41:16.060588 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:41:16.061450 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:41:16.061464 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:41:16.062267 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:41:16.062271 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:41:16.062386 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:41:16.062927 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:41:16.062940 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:41:16.068426 (+ 5486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:41:16.071285 (+ 2859us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:41:16.071288 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:41:16.071290 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:41:16.071548 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:41:16.071905 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:41:16.071909 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:41:16.071911 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:41:16.072007 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:41:16.072830 (+ 823us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:41:16.072833 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:41:16.072945 (+ 112us) client_negotiation.cc:769] Sending connection context
0806 11:41:16.073079 (+ 134us) client_negotiation.cc:240] Negotiation successful
0806 11:41:16.073253 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152}
I20260806 11:41:16.073652 12980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:16.058069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45582 (local address 127.21.14.253:34937)
0806 11:41:16.058526 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:16.058537 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:16.058561 (+ 24us) server_negotiation.cc:407] Connection header received
0806 11:41:16.058602 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:16.058607 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:16.058809 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:16.059062 (+ 253us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:16.059624 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:16.060478 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:16.061591 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:16.062134 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:16.062250 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:41:16.068612 (+ 6362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:41:16.068628 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:41:16.068646 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:41:16.068678 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:41:16.071150 (+ 2472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:41:16.071685 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:41:16.071693 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:41:16.071695 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:41:16.071789 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:41:16.072259 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:41:16.072262 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:41:16.072263 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:41:16.072617 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:41:16.072753 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 11:41:16.073209 (+ 456us) server_negotiation.cc:299] Negotiation successful
0806 11:41:16.073433 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":223,"threads_started":1}
I20260806 11:41:16.075784 12898 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.14.252" port: 33185 } has no permanent_uuid. Determining permanent_uuid...
I20260806 11:41:16.076608 12902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:16.076328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.252:33185 (local address 127.0.0.1:49152)
0806 11:41:16.076456 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0806 11:41:16.076517 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.252:33185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260806 11:41:16.077596 12902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:16.077210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.252:33185 (local address 127.0.0.1:49154)
0806 11:41:16.077423 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0806 11:41:16.077493 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.252:33185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
Aug 06 11:41:16 dist-test-slave-7c35 krb5kdc[12815](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016476, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.253@KRBTEST.COM for kudu/127.21.14.253@KRBTEST.COM
W20260806 11:41:16.077792 12898 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.14.252:33185: Network error: Client connection negotiation failed: client connection to 127.21.14.252:33185: connect: Connection refused (error 111)
I20260806 11:41:16.082129 12978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:16.072557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.253:34937 (local address 127.0.0.1:45588)
0806 11:41:16.072672 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0806 11:41:16.072688 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 11:41:16.072779 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:41:16.073186 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:41:16.073189 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:41:16.073409 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:41:16.073514 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:41:16.073521 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:41:16.074546 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:41:16.074553 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:41:16.075150 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:41:16.075159 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:41:16.075776 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:41:16.075779 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:41:16.075841 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:41:16.076532 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:41:16.076547 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:41:16.078013 (+ 1466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:41:16.080556 (+ 2543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:41:16.080560 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:41:16.080563 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:41:16.080761 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:41:16.081052 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:41:16.081057 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:41:16.081059 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:41:16.081143 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:41:16.081535 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:41:16.081538 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:41:16.081635 (+ 97us) client_negotiation.cc:769] Sending connection context
0806 11:41:16.081745 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 11:41:16.081895 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260806 11:41:16.082144 12982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:16.072626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45588 (local address 127.21.14.253:34937)
0806 11:41:16.072958 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:16.072967 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:16.072980 (+ 13us) server_negotiation.cc:407] Connection header received
0806 11:41:16.073016 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:16.073020 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:16.073070 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:16.073155 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:16.073673 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:16.074450 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:16.075268 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:16.075550 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:16.075657 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:41:16.078191 (+ 2534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:41:16.078195 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:41:16.078196 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:41:16.078216 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:41:16.080433 (+ 2217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:41:16.080864 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:41:16.080870 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:41:16.080874 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:41:16.080957 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:41:16.081235 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:41:16.081239 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:41:16.081241 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:41:16.081432 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:41:16.081549 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 11:41:16.081790 (+ 241us) server_negotiation.cc:299] Negotiation successful
0806 11:41:16.081956 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":63,"server-negotiator.queue_time_us":264,"thread_start_us":112,"threads_started":1}
I20260806 11:41:16.083029 12976 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.14.252" port: 33185 } has no permanent_uuid. Determining permanent_uuid...
I20260806 11:41:16.083797 12978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:16.083515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.252:33185 (local address 127.0.0.1:49166)
0806 11:41:16.083644 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0806 11:41:16.083736 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.252:33185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260806 11:41:16.084030 12924 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.14.252:33185: connect: Connection refused (error 111)
I20260806 11:41:16.085029 12978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:16.084801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.252:33185 (local address 127.0.0.1:49180)
0806 11:41:16.084911 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0806 11:41:16.084963 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.252:33185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260806 11:41:16.085178 12976 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.14.252:33185: Network error: Client connection negotiation failed: client connection to 127.21.14.252:33185: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.252.keytab.
Entry for principal kudu/127.21.14.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.252.keytab.
I20260806 11:41:16.091428 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.252:33185
--webserver_interface=127.21.14.252
--webserver_port=0
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.252.keytab
--principal=kudu/127.21.14.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:33535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.14.254:35671,127.21.14.253:34937,127.21.14.252:33185
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 11:41:16.109025 12898 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.14.252" port: 33185 } attempt: 1
I20260806 11:41:16.110047 12902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:16.109706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.252:33185 (local address 127.0.0.1:49194)
0806 11:41:16.109884 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 11:41:16.109956 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.252:33185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260806 11:41:16.110991 12902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:16.110683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.252:33185 (local address 127.0.0.1:49206)
0806 11:41:16.110806 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0806 11:41:16.110913 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.252:33185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260806 11:41:16.111141 12898 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.14.252:33185: Network error: Client connection negotiation failed: client connection to 127.21.14.252:33185: connect: Connection refused (error 111)
I20260806 11:41:16.134495 12976 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.14.252" port: 33185 } attempt: 1
I20260806 11:41:16.135556 12978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:16.135198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.252:33185 (local address 127.0.0.1:49218)
0806 11:41:16.135399 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0806 11:41:16.135470 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.252:33185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260806 11:41:16.136413 12978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:16.136165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.252:33185 (local address 127.0.0.1:49220)
0806 11:41:16.136289 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 11:41:16.136347 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.252:33185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260806 11:41:16.136550 12976 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.14.252:33185: Network error: Client connection negotiation failed: client connection to 127.21.14.252:33185: connect: Connection refused (error 111)
I20260806 11:41:16.186333 12898 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.14.252" port: 33185 } attempt: 2
I20260806 11:41:16.187309 12902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:16.186993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.252:33185 (local address 127.0.0.1:49226)
0806 11:41:16.187148 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 11:41:16.187222 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.252:33185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260806 11:41:16.188191 12902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:16.187946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.252:33185 (local address 127.0.0.1:49230)
0806 11:41:16.188069 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0806 11:41:16.188126 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.252:33185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260806 11:41:16.188330 12898 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.14.252:33185: Network error: Client connection negotiation failed: client connection to 127.21.14.252:33185: connect: Connection refused (error 111)
I20260806 11:41:16.204746 12976 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.14.252" port: 33185 } attempt: 2
I20260806 11:41:16.205715 12978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:16.205370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.252:33185 (local address 127.0.0.1:49244)
0806 11:41:16.205566 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0806 11:41:16.205634 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.252:33185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260806 11:41:16.206547 12978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:16.206299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.252:33185 (local address 127.0.0.1:49252)
0806 11:41:16.206427 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0806 11:41:16.206483 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.252:33185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260806 11:41:16.206681 12976 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.14.252:33185: Network error: Client connection negotiation failed: client connection to 127.21.14.252:33185: connect: Connection refused (error 111)
W20260806 11:41:16.217151 12984 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 11:41:16.217516 12984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:41:16.217588 12984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:41:16.217691 12984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:41:16.222082 12984 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 11:41:16.222172 12984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:41:16.222209 12984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:41:16.222268 12984 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 11:41:16.222306 12984 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 11:41:16.227977 12984 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:33535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.21.14.254:35671,127.21.14.253:34937,127.21.14.252:33185
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.252.keytab
--principal=kudu/127.21.14.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.252:33185
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.21.14.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12984
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:41:16.229270 12984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:41:16.230429 12984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:41:16.239600 12992 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:41:16.239611 12990 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:41:16.239600 12989 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:41:16.240001 12984 server_base.cc:1034] running on GCE node
I20260806 11:41:16.240657 12984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:41:16.241888 12984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:41:16.243060 12984 hybrid_clock.cc:648] HybridClock initialized: now 1786016476243046 us; error 29 us; skew 500 ppm
Aug 06 11:41:16 dist-test-slave-7c35 krb5kdc[12815](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016476, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:41:16.246172 12984 init.cc:377] Logged in from keytab as kudu/127.21.14.252@KRBTEST.COM (short username kudu)
I20260806 11:41:16.247301 12984 webserver.cc:460] Webserver started at http://127.21.14.252:39185/ using document root <none> and password file <none>
I20260806 11:41:16.247917 12984 fs_manager.cc:359] Metadata directory not provided
I20260806 11:41:16.248016 12984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:41:16.248273 12984 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:41:16.250499 12984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/minicluster-data/master-2/data/instance:
uuid: "897c2aff10254c7ebbb243fe7f37af67"
format_stamp: "Formatted at 2026-08-06 11:41:16 on dist-test-slave-7c35"
server_key: "7cbf8d6f5b1a7678f4323b88974a0edb"
server_key_iv: "c82828d8221201f395304c45ae369578"
server_key_version: "encryptionkey@0"
I20260806 11:41:16.251133 12984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/minicluster-data/master-2/wal/instance:
uuid: "897c2aff10254c7ebbb243fe7f37af67"
format_stamp: "Formatted at 2026-08-06 11:41:16 on dist-test-slave-7c35"
server_key: "7cbf8d6f5b1a7678f4323b88974a0edb"
server_key_iv: "c82828d8221201f395304c45ae369578"
server_key_version: "encryptionkey@0"
I20260806 11:41:16.255183 12984 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 11:41:16.257910 12999 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:41:16.259375 12984 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 11:41:16.259532 12984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/minicluster-data/master-2/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/minicluster-data/master-2/wal
uuid: "897c2aff10254c7ebbb243fe7f37af67"
format_stamp: "Formatted at 2026-08-06 11:41:16 on dist-test-slave-7c35"
server_key: "7cbf8d6f5b1a7678f4323b88974a0edb"
server_key_iv: "c82828d8221201f395304c45ae369578"
server_key_version: "encryptionkey@0"
I20260806 11:41:16.259667 12984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:41:16.272639 12984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:41:16.276480 12984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:41:16.276722 12984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:41:16.284616 12984 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.252:33185
I20260806 11:41:16.284915 13052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.252:33185 every 8 connection(s)
I20260806 11:41:16.285784 12984 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/minicluster-data/master-2/data/info.pb
I20260806 11:41:16.287501 12898 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.14.252" port: 33185 } attempt: 3
I20260806 11:41:16.289248 13053 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:41:16.289873 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 12984
I20260806 11:41:16.289999 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786016169045816-21563-0/minicluster-data/master-2/wal/instance
I20260806 11:41:16.290336 21563 external_mini_cluster.cc:1428] Setting key 5695a74571305c52de1811a2bd6024f1
I20260806 11:41:16.294962 13053 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.14.254" port: 35671 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 11:41:16 dist-test-slave-7c35 krb5kdc[12815](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016475, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.254@KRBTEST.COM for kudu/127.21.14.252@KRBTEST.COM
Aug 06 11:41:16 dist-test-slave-7c35 krb5kdc[12815](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016475, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.14.252@KRBTEST.COM
I20260806 11:41:16.297977 12976 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.14.252" port: 33185 } attempt: 3
I20260806 11:41:16.303795 12902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:16.288137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.252:33185 (local address 127.0.0.1:49268)
0806 11:41:16.288314 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 11:41:16.288334 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 11:41:16.288439 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:41:16.291522 (+ 3083us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:41:16.291526 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:41:16.291771 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:41:16.291895 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:41:16.291903 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:41:16.293428 (+ 1525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:41:16.293432 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:41:16.294258 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:41:16.294277 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:41:16.295310 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:41:16.295315 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:41:16.295418 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:41:16.296152 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:41:16.296168 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:41:16.297838 (+ 1670us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:41:16.301574 (+ 3736us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:41:16.301578 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:41:16.301580 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:41:16.301850 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:41:16.302577 (+ 727us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:41:16.302581 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:41:16.302584 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:41:16.302708 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:41:16.303200 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:41:16.303203 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:41:16.303308 (+ 105us) client_negotiation.cc:769] Sending connection context
0806 11:41:16.303449 (+ 141us) client_negotiation.cc:240] Negotiation successful
0806 11:41:16.303617 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
Aug 06 11:41:16 dist-test-slave-7c35 krb5kdc[12815](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016476, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.252@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
Aug 06 11:41:16 dist-test-slave-7c35 krb5kdc[12815](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016476, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.253@KRBTEST.COM for kudu/127.21.14.252@KRBTEST.COM
I20260806 11:41:16.305625 13057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:16.291057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49274 (local address 127.21.14.252:33185)
0806 11:41:16.291393 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:16.291402 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:16.291740 (+ 338us) server_negotiation.cc:407] Connection header received
0806 11:41:16.291780 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:16.291783 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:16.291826 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:16.291904 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:16.293080 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:16.293713 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:16.294707 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:16.295142 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:16.295315 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:41:16.298986 (+ 3671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:41:16.298992 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:41:16.298998 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:41:16.299027 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:41:16.301414 (+ 2387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:41:16.301953 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:41:16.301959 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:41:16.301965 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:41:16.302123 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:41:16.302467 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:41:16.302470 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:41:16.302472 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:41:16.303008 (+ 536us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:41:16.303219 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0806 11:41:16.303808 (+ 589us) server_negotiation.cc:299] Negotiation successful
0806 11:41:16.304530 (+ 722us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":140,"server-negotiator.queue_time_us":242,"thread_start_us":132,"threads_started":1}
I20260806 11:41:16.305625 13055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:16.288982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49268 (local address 127.21.14.252:33185)
0806 11:41:16.289489 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:16.289514 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:16.289567 (+ 53us) server_negotiation.cc:407] Connection header received
0806 11:41:16.290608 (+ 1041us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:16.290636 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:16.291121 (+ 485us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:16.291670 (+ 549us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:16.292157 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:16.293257 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:16.294460 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:16.295150 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:16.295700 (+ 550us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:41:16.298035 (+ 2335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:41:16.298041 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:41:16.298057 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:41:16.298092 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:41:16.301448 (+ 3356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:41:16.301975 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:41:16.301978 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:41:16.301979 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:41:16.302136 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:41:16.302823 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:41:16.302827 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:41:16.302829 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:41:16.303106 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:41:16.303207 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 11:41:16.304160 (+ 953us) server_negotiation.cc:299] Negotiation successful
0806 11:41:16.304478 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":53,"server-negotiator.queue_time_us":336,"spinlock_wait_cycles":2304,"thread_start_us":160,"threads_started":1}
I20260806 11:41:16.309866 12901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:16.297202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37884 (local address 127.21.14.254:35671)
0806 11:41:16.297339 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:16.297344 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:16.297728 (+ 384us) server_negotiation.cc:407] Connection header received
0806 11:41:16.297948 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:16.297951 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:16.298002 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:16.298061 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:16.299109 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:16.299884 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:16.301278 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:16.301619 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:16.301700 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:41:16.305788 (+ 4088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:41:16.305793 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:41:16.305796 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:41:16.305826 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:41:16.307809 (+ 1983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:41:16.308374 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:41:16.308377 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:41:16.308379 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:41:16.308482 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:41:16.308827 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:41:16.308831 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:41:16.308833 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:41:16.309124 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:41:16.309190 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 11:41:16.309575 (+ 385us) server_negotiation.cc:299] Negotiation successful
0806 11:41:16.309704 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 11:41:16.309880 13058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:16.297134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:35671 (local address 127.0.0.1:37884)
0806 11:41:16.297588 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0806 11:41:16.297646 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 11:41:16.297831 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:41:16.298256 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:41:16.298266 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:41:16.298758 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:41:16.298962 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:41:16.298970 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:41:16.300038 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:41:16.300042 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:41:16.301122 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:41:16.301135 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:41:16.301876 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:41:16.301881 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:41:16.301996 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:41:16.303322 (+ 1326us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:41:16.303341 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:41:16.305639 (+ 2298us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:41:16.307947 (+ 2308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:41:16.307958 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:41:16.307963 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:41:16.308258 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:41:16.308599 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:41:16.308603 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:41:16.308605 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:41:16.308714 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:41:16.309264 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:41:16.309269 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:41:16.309369 (+ 100us) client_negotiation.cc:769] Sending connection context
0806 11:41:16.309518 (+ 149us) client_negotiation.cc:240] Negotiation successful
0806 11:41:16.309704 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":132,"threads_started":1}
I20260806 11:41:16.311247 13053 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.14.253" port: 34937 } has no permanent_uuid. Determining permanent_uuid...
I20260806 11:41:16.311295 12898 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b300ea3e113d4528911b80e89f04bfdd: Bootstrap starting.
I20260806 11:41:16.312335 12978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:16.298422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.252:33185 (local address 127.0.0.1:49288)
0806 11:41:16.298565 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 11:41:16.298582 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 11:41:16.298670 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:41:16.299405 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:41:16.299410 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:41:16.299641 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:41:16.299770 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:41:16.299792 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:41:16.300913 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:41:16.300918 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:41:16.301768 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:41:16.301782 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:41:16.303061 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:41:16.303074 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:41:16.303172 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:41:16.304178 (+ 1006us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:41:16.304194 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:41:16.305525 (+ 1331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:41:16.307726 (+ 2201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:41:16.307730 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:41:16.307732 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:41:16.307998 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:41:16.311168 (+ 3170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:41:16.311171 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:41:16.311173 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:41:16.311285 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:41:16.311776 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:41:16.311781 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:41:16.311899 (+ 118us) client_negotiation.cc:769] Sending connection context
0806 11:41:16.312015 (+ 116us) client_negotiation.cc:240] Negotiation successful
0806 11:41:16.312176 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260806 11:41:16.312368 13059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:16.298774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49288 (local address 127.21.14.252:33185)
0806 11:41:16.299146 (+ 372us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:16.299158 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:16.299174 (+ 16us) server_negotiation.cc:407] Connection header received
0806 11:41:16.299212 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:16.299214 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:16.299261 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:16.299362 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:16.299934 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:16.300767 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:16.301926 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:16.302928 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:16.303031 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:41:16.305695 (+ 2664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:41:16.305699 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:41:16.305701 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:41:16.305727 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:41:16.307603 (+ 1876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:41:16.310940 (+ 3337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:41:16.310943 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:41:16.310945 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:41:16.311041 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:41:16.311419 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:41:16.311422 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:41:16.311424 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:41:16.311645 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:41:16.311770 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 11:41:16.312070 (+ 300us) server_negotiation.cc:299] Negotiation successful
0806 11:41:16.312184 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":297,"thread_start_us":135,"threads_started":1}
I20260806 11:41:16.314608 12898 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b300ea3e113d4528911b80e89f04bfdd: Neither blocks nor log segments found. Creating new log.
Aug 06 11:41:16 dist-test-slave-7c35 krb5kdc[12815](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016475, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:41:16.315698 12976 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e6a50bf43cd4cbcad84c8975b5542a3: Bootstrap starting.
I20260806 11:41:16.316267 12898 log.cc:824] T 00000000000000000000000000000000 P b300ea3e113d4528911b80e89f04bfdd: Log is configured to *not* fsync() on all Append() calls
Aug 06 11:41:16 dist-test-slave-7c35 krb5kdc[12815](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016476, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.252@KRBTEST.COM for kudu/127.21.14.253@KRBTEST.COM
I20260806 11:41:16.318631 12976 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3e6a50bf43cd4cbcad84c8975b5542a3: Neither blocks nor log segments found. Creating new log.
I20260806 11:41:16.318758 12898 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b300ea3e113d4528911b80e89f04bfdd: No bootstrap required, opened a new log
I20260806 11:41:16.319664 12976 log.cc:824] T 00000000000000000000000000000000 P 3e6a50bf43cd4cbcad84c8975b5542a3: Log is configured to *not* fsync() on all Append() calls
I20260806 11:41:16.321210 13060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:16.308840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37890 (local address 127.21.14.254:35671)
0806 11:41:16.309142 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:16.309155 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:16.309168 (+ 13us) server_negotiation.cc:407] Connection header received
0806 11:41:16.309199 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:16.309201 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:16.309240 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:16.310159 (+ 919us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:16.310259 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:16.311040 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:16.312727 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:16.313096 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:16.315273 (+ 2177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:41:16.316342 (+ 1069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:41:16.316347 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:41:16.316351 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:41:16.316379 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:41:16.318418 (+ 2039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:41:16.319272 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:41:16.319275 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:41:16.319277 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:41:16.319369 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:41:16.319708 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:41:16.319711 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:41:16.319712 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:41:16.319939 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:41:16.320771 (+ 832us) server_negotiation.cc:1035] Waiting for connection context
0806 11:41:16.320850 (+ 79us) server_negotiation.cc:299] Negotiation successful
0806 11:41:16.321002 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":117,"threads_started":1}
I20260806 11:41:16.322567 12982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:16.312502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45602 (local address 127.21.14.253:34937)
0806 11:41:16.312631 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:16.312636 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:16.313337 (+ 701us) server_negotiation.cc:407] Connection header received
0806 11:41:16.313423 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:16.313429 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:16.313482 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:16.313561 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:16.314143 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:16.314934 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:16.315992 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:16.316320 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:16.316406 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:41:16.318672 (+ 2266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:41:16.318676 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:41:16.318680 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:41:16.318707 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:41:16.320547 (+ 1840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:41:16.321065 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:41:16.321069 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:41:16.321070 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:41:16.321184 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:41:16.321512 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:41:16.321519 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:41:16.321523 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:41:16.321787 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:41:16.321886 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 11:41:16.322268 (+ 382us) server_negotiation.cc:299] Negotiation successful
0806 11:41:16.322412 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 11:41:16.322608 13058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:16.312253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.253:34937 (local address 127.0.0.1:45602)
0806 11:41:16.313254 (+ 1001us) negotiation.cc:107] Waiting for socket to connect
0806 11:41:16.313269 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 11:41:16.313347 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:41:16.313680 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:41:16.313683 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:41:16.313844 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:41:16.314011 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:41:16.314019 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:41:16.315075 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:41:16.315079 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:41:16.315851 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:41:16.315862 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:41:16.316560 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:41:16.316563 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:41:16.316657 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:41:16.317032 (+ 375us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:41:16.317046 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:41:16.318502 (+ 1456us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:41:16.320687 (+ 2185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:41:16.320690 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:41:16.320692 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:41:16.320915 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:41:16.321279 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:41:16.321282 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:41:16.321285 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:41:16.321407 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:41:16.321966 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:41:16.321969 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:41:16.322094 (+ 125us) client_negotiation.cc:769] Sending connection context
0806 11:41:16.322203 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 11:41:16.322436 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":928}
I20260806 11:41:16.323227 12898 raft_consensus.cc:348] T 00000000000000000000000000000000 P b300ea3e113d4528911b80e89f04bfdd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b300ea3e113d4528911b80e89f04bfdd" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 35671 } } peers { permanent_uuid: "3e6a50bf43cd4cbcad84c8975b5542a3" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 34937 } } peers { permanent_uuid: "897c2aff10254c7ebbb243fe7f37af67" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 33185 } }
I20260806 11:41:16.323498 12898 raft_consensus.cc:729] T 00000000000000000000000000000000 P b300ea3e113d4528911b80e89f04bfdd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b300ea3e113d4528911b80e89f04bfdd, State: Initialized, Role: FOLLOWER
I20260806 11:41:16.323555 13053 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.14.252" port: 33185 } has no permanent_uuid. Determining permanent_uuid...
I20260806 11:41:16.323912 12898 consensus_queue.cc:260] T 00000000000000000000000000000000 P b300ea3e113d4528911b80e89f04bfdd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b300ea3e113d4528911b80e89f04bfdd" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 35671 } } peers { permanent_uuid: "3e6a50bf43cd4cbcad84c8975b5542a3" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 34937 } } peers { permanent_uuid: "897c2aff10254c7ebbb243fe7f37af67" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 33185 } }
I20260806 11:41:16.324436 13063 sys_catalog.cc:455] T 00000000000000000000000000000000 P b300ea3e113d4528911b80e89f04bfdd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b300ea3e113d4528911b80e89f04bfdd" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 35671 } } peers { permanent_uuid: "3e6a50bf43cd4cbcad84c8975b5542a3" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 34937 } } peers { permanent_uuid: "897c2aff10254c7ebbb243fe7f37af67" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 33185 } } }
I20260806 11:41:16.324573 13063 sys_catalog.cc:458] T 00000000000000000000000000000000 P b300ea3e113d4528911b80e89f04bfdd [sys.catalog]: This master's current role is: FOLLOWER
I20260806 11:41:16.325426 12898 sys_catalog.cc:564] T 00000000000000000000000000000000 P b300ea3e113d4528911b80e89f04bfdd [sys.catalog]: configured and running, proceeding with master startup.
Aug 06 11:41:16 dist-test-slave-7c35 krb5kdc[12815](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016475, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.14.253@KRBTEST.COM
Aug 06 11:41:16 dist-test-slave-7c35 krb5kdc[12815](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016476, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.252@KRBTEST.COM for kudu/127.21.14.252@KRBTEST.COM
I20260806 11:41:16.332170 12976 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e6a50bf43cd4cbcad84c8975b5542a3: No bootstrap required, opened a new log
W20260806 11:41:16.332604 13074 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b300ea3e113d4528911b80e89f04bfdd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 11:41:16.332708 13074 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 11:41:16.330787 12828 master_runner.cc:186] Error getting master registration for 127.21.14.253:34937: OK, instance_id { permanent_uuid: "3e6a50bf43cd4cbcad84c8975b5542a3" instance_seqno: 1786016476041082 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 11:41:16.334261 12828 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 11:41:16.336659 12980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:16.322591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45606 (local address 127.21.14.253:34937)
0806 11:41:16.322710 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:16.322715 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:16.323217 (+ 502us) server_negotiation.cc:407] Connection header received
0806 11:41:16.323334 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:16.323336 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:16.323403 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:16.323482 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:16.324217 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:16.325031 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:16.326072 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:16.326442 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:16.326607 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:41:16.331185 (+ 4578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:41:16.331190 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:41:16.331194 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:41:16.331232 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:41:16.333509 (+ 2277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:41:16.334216 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:41:16.334221 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:41:16.334224 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:41:16.334334 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:41:16.334771 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:41:16.334775 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:41:16.334777 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:41:16.335105 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:41:16.335351 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0806 11:41:16.336293 (+ 942us) server_negotiation.cc:299] Negotiation successful
0806 11:41:16.336455 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 11:41:16.336889 12976 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3e6a50bf43cd4cbcad84c8975b5542a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b300ea3e113d4528911b80e89f04bfdd" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 35671 } } peers { permanent_uuid: "3e6a50bf43cd4cbcad84c8975b5542a3" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 34937 } } peers { permanent_uuid: "897c2aff10254c7ebbb243fe7f37af67" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 33185 } }
I20260806 11:41:16.337167 12976 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3e6a50bf43cd4cbcad84c8975b5542a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e6a50bf43cd4cbcad84c8975b5542a3, State: Initialized, Role: FOLLOWER
I20260806 11:41:16.337531 12976 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3e6a50bf43cd4cbcad84c8975b5542a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b300ea3e113d4528911b80e89f04bfdd" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 35671 } } peers { permanent_uuid: "3e6a50bf43cd4cbcad84c8975b5542a3" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 34937 } } peers { permanent_uuid: "897c2aff10254c7ebbb243fe7f37af67" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 33185 } }
I20260806 11:41:16.338291 13076 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e6a50bf43cd4cbcad84c8975b5542a3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b300ea3e113d4528911b80e89f04bfdd" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 35671 } } peers { permanent_uuid: "3e6a50bf43cd4cbcad84c8975b5542a3" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 34937 } } peers { permanent_uuid: "897c2aff10254c7ebbb243fe7f37af67" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 33185 } } }
I20260806 11:41:16.338542 13076 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e6a50bf43cd4cbcad84c8975b5542a3 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 11:41:16.339356 12976 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3e6a50bf43cd4cbcad84c8975b5542a3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 11:41:16.339727 13058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:16.324028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.252:33185 (local address 127.0.0.1:49300)
0806 11:41:16.324416 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0806 11:41:16.324429 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 11:41:16.324525 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:41:16.324790 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:41:16.324792 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:41:16.324951 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:41:16.325021 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:41:16.325025 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:41:16.327040 (+ 2015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:41:16.327046 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:41:16.327919 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:41:16.327934 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:41:16.328517 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:41:16.328520 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:41:16.328594 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:41:16.329097 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:41:16.329111 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:41:16.334637 (+ 5526us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:41:16.336741 (+ 2104us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:41:16.336744 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:41:16.336746 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:41:16.336995 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:41:16.337662 (+ 667us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:41:16.337677 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:41:16.337679 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:41:16.337802 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:41:16.339245 (+ 1443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:41:16.339248 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:41:16.339330 (+ 82us) client_negotiation.cc:769] Sending connection context
0806 11:41:16.339414 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 11:41:16.339565 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332}
I20260806 11:41:16.340394 13059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:16.324245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49300 (local address 127.21.14.252:33185)
0806 11:41:16.324346 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:16.324351 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:16.324474 (+ 123us) server_negotiation.cc:407] Connection header received
0806 11:41:16.324606 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:16.324609 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:16.324647 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:16.324689 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:16.326087 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:16.326927 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:16.328058 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:16.328399 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:16.329722 (+ 1323us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:41:16.334795 (+ 5073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:41:16.334799 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:41:16.334802 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:41:16.334825 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:41:16.336622 (+ 1797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:41:16.337123 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:41:16.337126 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:41:16.337128 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:41:16.337470 (+ 342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:41:16.338900 (+ 1430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:41:16.338903 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:41:16.338905 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:41:16.339135 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:41:16.340042 (+ 907us) server_negotiation.cc:1035] Waiting for connection context
0806 11:41:16.340117 (+ 75us) server_negotiation.cc:299] Negotiation successful
0806 11:41:16.340252 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 11:41:16.344040 13053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 897c2aff10254c7ebbb243fe7f37af67: Bootstrap starting.
W20260806 11:41:16.346377 13089 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3e6a50bf43cd4cbcad84c8975b5542a3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 11:41:16.346454 13089 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 11:41:16.347298 13053 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 897c2aff10254c7ebbb243fe7f37af67: Neither blocks nor log segments found. Creating new log.
I20260806 11:41:16.348323 13053 log.cc:824] T 00000000000000000000000000000000 P 897c2aff10254c7ebbb243fe7f37af67: Log is configured to *not* fsync() on all Append() calls
I20260806 11:41:16.349110 21563 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 11:41:16.350998 12872 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "b300ea3e113d4528911b80e89f04bfdd"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37890
I20260806 11:41:16.351352 12872 raft_consensus.cc:482] T 00000000000000000000000000000000 P b300ea3e113d4528911b80e89f04bfdd [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 11:41:16.351492 12872 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b300ea3e113d4528911b80e89f04bfdd [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:41:16.349524 12905 master_runner.cc:186] Error getting master registration for 127.21.14.252:33185: OK, instance_id { permanent_uuid: "897c2aff10254c7ebbb243fe7f37af67" instance_seqno: 1786016476282296 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 11:41:16.352947 12905 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 11:41:16.353150 12872 raft_consensus.cc:504] T 00000000000000000000000000000000 P b300ea3e113d4528911b80e89f04bfdd [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b300ea3e113d4528911b80e89f04bfdd" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 35671 } } peers { permanent_uuid: "3e6a50bf43cd4cbcad84c8975b5542a3" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 34937 } } peers { permanent_uuid: "897c2aff10254c7ebbb243fe7f37af67" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 33185 } }
I20260806 11:41:16.355124 12949 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b300ea3e113d4528911b80e89f04bfdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "3e6a50bf43cd4cbcad84c8975b5542a3"
I20260806 11:41:16.355396 12949 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3e6a50bf43cd4cbcad84c8975b5542a3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:41:16.355242 12872 leader_election.cc:288] T 00000000000000000000000000000000 P b300ea3e113d4528911b80e89f04bfdd [CANDIDATE]: Term 1 election: Requested vote from peers 3e6a50bf43cd4cbcad84c8975b5542a3 (127.21.14.253:34937), 897c2aff10254c7ebbb243fe7f37af67 (127.21.14.252:33185)
I20260806 11:41:16.356829 12949 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3e6a50bf43cd4cbcad84c8975b5542a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b300ea3e113d4528911b80e89f04bfdd in term 1.
I20260806 11:41:16.357316 12846 leader_election.cc:302] T 00000000000000000000000000000000 P b300ea3e113d4528911b80e89f04bfdd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e6a50bf43cd4cbcad84c8975b5542a3, b300ea3e113d4528911b80e89f04bfdd; no voters:
I20260806 11:41:16.357678 13063 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b300ea3e113d4528911b80e89f04bfdd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:41:16.357914 13063 raft_consensus.cc:686] T 00000000000000000000000000000000 P b300ea3e113d4528911b80e89f04bfdd [term 1 LEADER]: Becoming Leader. State: Replica: b300ea3e113d4528911b80e89f04bfdd, State: Running, Role: LEADER
I20260806 11:41:16.358304 13063 consensus_queue.cc:237] T 00000000000000000000000000000000 P b300ea3e113d4528911b80e89f04bfdd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b300ea3e113d4528911b80e89f04bfdd" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 35671 } } peers { permanent_uuid: "3e6a50bf43cd4cbcad84c8975b5542a3" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 34937 } } peers { permanent_uuid: "897c2aff10254c7ebbb243fe7f37af67" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 33185 } }
I20260806 11:41:16.361589 13053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 897c2aff10254c7ebbb243fe7f37af67: No bootstrap required, opened a new log
I20260806 11:41:16.363770 13096 sys_catalog.cc:455] T 00000000000000000000000000000000 P b300ea3e113d4528911b80e89f04bfdd [sys.catalog]: SysCatalogTable state changed. Reason: New leader b300ea3e113d4528911b80e89f04bfdd. Latest consensus state: current_term: 1 leader_uuid: "b300ea3e113d4528911b80e89f04bfdd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b300ea3e113d4528911b80e89f04bfdd" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 35671 } } peers { permanent_uuid: "3e6a50bf43cd4cbcad84c8975b5542a3" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 34937 } } peers { permanent_uuid: "897c2aff10254c7ebbb243fe7f37af67" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 33185 } } }
I20260806 11:41:16.364020 13096 sys_catalog.cc:458] T 00000000000000000000000000000000 P b300ea3e113d4528911b80e89f04bfdd [sys.catalog]: This master's current role is: LEADER
I20260806 11:41:16.367800 13053 raft_consensus.cc:348] T 00000000000000000000000000000000 P 897c2aff10254c7ebbb243fe7f37af67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b300ea3e113d4528911b80e89f04bfdd" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 35671 } } peers { permanent_uuid: "3e6a50bf43cd4cbcad84c8975b5542a3" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 34937 } } peers { permanent_uuid: "897c2aff10254c7ebbb243fe7f37af67" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 33185 } }
I20260806 11:41:16.375275 13101 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 11:41:16.376636 13101 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 11:41:16.376844 13053 raft_consensus.cc:729] T 00000000000000000000000000000000 P 897c2aff10254c7ebbb243fe7f37af67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 897c2aff10254c7ebbb243fe7f37af67, State: Initialized, Role: FOLLOWER
I20260806 11:41:16.378077 13053 consensus_queue.cc:260] T 00000000000000000000000000000000 P 897c2aff10254c7ebbb243fe7f37af67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b300ea3e113d4528911b80e89f04bfdd" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 35671 } } peers { permanent_uuid: "3e6a50bf43cd4cbcad84c8975b5542a3" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 34937 } } peers { permanent_uuid: "897c2aff10254c7ebbb243fe7f37af67" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 33185 } }
I20260806 11:41:16.380424 13053 sys_catalog.cc:564] T 00000000000000000000000000000000 P 897c2aff10254c7ebbb243fe7f37af67 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 11:41:16.380529 13025 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b300ea3e113d4528911b80e89f04bfdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "897c2aff10254c7ebbb243fe7f37af67"
I20260806 11:41:16.380877 13025 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 897c2aff10254c7ebbb243fe7f37af67 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:41:16.382367 13025 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 897c2aff10254c7ebbb243fe7f37af67 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b300ea3e113d4528911b80e89f04bfdd in term 1.
I20260806 11:41:16.383569 13102 sys_catalog.cc:455] T 00000000000000000000000000000000 P 897c2aff10254c7ebbb243fe7f37af67 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b300ea3e113d4528911b80e89f04bfdd" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 35671 } } peers { permanent_uuid: "3e6a50bf43cd4cbcad84c8975b5542a3" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 34937 } } peers { permanent_uuid: "897c2aff10254c7ebbb243fe7f37af67" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 33185 } } }
I20260806 11:41:16.385077 13102 sys_catalog.cc:458] T 00000000000000000000000000000000 P 897c2aff10254c7ebbb243fe7f37af67 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 11:41:16.386610 12980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:16.360046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45616 (local address 127.21.14.253:34937)
0806 11:41:16.360171 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:16.360176 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:16.361146 (+ 970us) server_negotiation.cc:407] Connection header received
0806 11:41:16.361470 (+ 324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:16.361474 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:16.361528 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:16.361582 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:16.364919 (+ 3337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:16.365769 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:16.367669 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:16.368037 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:16.368115 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:41:16.377203 (+ 9088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:41:16.377209 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:41:16.377213 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:41:16.377281 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:41:16.384045 (+ 6764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:41:16.384746 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:41:16.384750 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:41:16.384752 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:41:16.384854 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:41:16.385204 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:41:16.385208 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:41:16.385209 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:41:16.385486 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:41:16.385625 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 11:41:16.386308 (+ 683us) server_negotiation.cc:299] Negotiation successful
0806 11:41:16.386450 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 11:41:16.389175 13060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:16.360643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37902 (local address 127.21.14.254:35671)
0806 11:41:16.360758 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:16.360763 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:16.368543 (+ 7780us) server_negotiation.cc:407] Connection header received
0806 11:41:16.368773 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:16.368778 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:16.368852 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:16.368918 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:16.370063 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:16.370947 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:16.372670 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:16.373081 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:16.373161 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:41:16.377906 (+ 4745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:41:16.377910 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:41:16.377913 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:41:16.377942 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:41:16.380366 (+ 2424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:41:16.387616 (+ 7250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:41:16.387620 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:41:16.387622 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:41:16.387731 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:41:16.388086 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:41:16.388090 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:41:16.388091 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:41:16.388351 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:41:16.388479 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 11:41:16.388885 (+ 406us) server_negotiation.cc:299] Negotiation successful
0806 11:41:16.389026 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
W20260806 11:41:16.390452 13114 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 897c2aff10254c7ebbb243fe7f37af67: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 11:41:16.390874 13114 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 11:41:16.392563 13059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:16.361122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49314 (local address 127.21.14.252:33185)
0806 11:41:16.361260 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:16.361266 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:16.369266 (+ 8000us) server_negotiation.cc:407] Connection header received
0806 11:41:16.369529 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:16.369535 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:16.369609 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:16.369683 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:16.371662 (+ 1979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:16.372572 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:16.373940 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:16.374345 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:16.374439 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:41:16.387182 (+ 12743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:41:16.387186 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:41:16.387189 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:41:16.387219 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:41:16.389041 (+ 1822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:41:16.390926 (+ 1885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:41:16.390929 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:41:16.390932 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:41:16.391021 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:41:16.391320 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:41:16.391323 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:41:16.391325 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:41:16.391539 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:41:16.391680 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 11:41:16.392291 (+ 611us) server_negotiation.cc:299] Negotiation successful
0806 11:41:16.392416 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 11:41:16.400907 12949 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3e6a50bf43cd4cbcad84c8975b5542a3 [term 1 FOLLOWER]: Refusing update from remote peer b300ea3e113d4528911b80e89f04bfdd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 11:41:16.400907 13025 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 897c2aff10254c7ebbb243fe7f37af67 [term 1 FOLLOWER]: Refusing update from remote peer b300ea3e113d4528911b80e89f04bfdd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 11:41:16.401592 13096 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b300ea3e113d4528911b80e89f04bfdd [LEADER]: Connected to new peer: Peer: permanent_uuid: "897c2aff10254c7ebbb243fe7f37af67" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 11:41:16.402128 13096 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b300ea3e113d4528911b80e89f04bfdd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e6a50bf43cd4cbcad84c8975b5542a3" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 34937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 11:41:16.406469 13102 sys_catalog.cc:455] T 00000000000000000000000000000000 P 897c2aff10254c7ebbb243fe7f37af67 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b300ea3e113d4528911b80e89f04bfdd. Latest consensus state: current_term: 1 leader_uuid: "b300ea3e113d4528911b80e89f04bfdd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b300ea3e113d4528911b80e89f04bfdd" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 35671 } } peers { permanent_uuid: "3e6a50bf43cd4cbcad84c8975b5542a3" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 34937 } } peers { permanent_uuid: "897c2aff10254c7ebbb243fe7f37af67" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 33185 } } }
I20260806 11:41:16.406612 13102 sys_catalog.cc:458] T 00000000000000000000000000000000 P 897c2aff10254c7ebbb243fe7f37af67 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 11:41:16.407248 13076 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e6a50bf43cd4cbcad84c8975b5542a3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b300ea3e113d4528911b80e89f04bfdd. Latest consensus state: current_term: 1 leader_uuid: "b300ea3e113d4528911b80e89f04bfdd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b300ea3e113d4528911b80e89f04bfdd" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 35671 } } peers { permanent_uuid: "3e6a50bf43cd4cbcad84c8975b5542a3" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 34937 } } peers { permanent_uuid: "897c2aff10254c7ebbb243fe7f37af67" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 33185 } } }
I20260806 11:41:16.407378 13076 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e6a50bf43cd4cbcad84c8975b5542a3 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 11:41:16.415349 13063 sys_catalog.cc:455] T 00000000000000000000000000000000 P b300ea3e113d4528911b80e89f04bfdd [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b300ea3e113d4528911b80e89f04bfdd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b300ea3e113d4528911b80e89f04bfdd" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 35671 } } peers { permanent_uuid: "3e6a50bf43cd4cbcad84c8975b5542a3" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 34937 } } peers { permanent_uuid: "897c2aff10254c7ebbb243fe7f37af67" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 33185 } } }
I20260806 11:41:16.415553 13063 sys_catalog.cc:458] T 00000000000000000000000000000000 P b300ea3e113d4528911b80e89f04bfdd [sys.catalog]: This master's current role is: LEADER
I20260806 11:41:16.415808 13076 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e6a50bf43cd4cbcad84c8975b5542a3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b300ea3e113d4528911b80e89f04bfdd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b300ea3e113d4528911b80e89f04bfdd" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 35671 } } peers { permanent_uuid: "3e6a50bf43cd4cbcad84c8975b5542a3" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 34937 } } peers { permanent_uuid: "897c2aff10254c7ebbb243fe7f37af67" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 33185 } } }
I20260806 11:41:16.415889 13103 mvcc.cc:205] Tried to move back new op lower bound from 7315523487312232448 to 7315523487178141696. Current Snapshot: MvccSnapshot[applied={T|T < 7315523487312232448}]
I20260806 11:41:16.415959 13076 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e6a50bf43cd4cbcad84c8975b5542a3 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 11:41:16.420126 13117 mvcc.cc:205] Tried to move back new op lower bound from 7315523487312232448 to 7315523487178141696. Current Snapshot: MvccSnapshot[applied={T|T < 7315523487312232448}]
I20260806 11:41:16.420444 13096 sys_catalog.cc:455] T 00000000000000000000000000000000 P b300ea3e113d4528911b80e89f04bfdd [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b300ea3e113d4528911b80e89f04bfdd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b300ea3e113d4528911b80e89f04bfdd" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 35671 } } peers { permanent_uuid: "3e6a50bf43cd4cbcad84c8975b5542a3" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 34937 } } peers { permanent_uuid: "897c2aff10254c7ebbb243fe7f37af67" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 33185 } } }
I20260806 11:41:16.420608 13096 sys_catalog.cc:458] T 00000000000000000000000000000000 P b300ea3e113d4528911b80e89f04bfdd [sys.catalog]: This master's current role is: LEADER
I20260806 11:41:16.422343 13102 sys_catalog.cc:455] T 00000000000000000000000000000000 P 897c2aff10254c7ebbb243fe7f37af67 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b300ea3e113d4528911b80e89f04bfdd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b300ea3e113d4528911b80e89f04bfdd" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 35671 } } peers { permanent_uuid: "3e6a50bf43cd4cbcad84c8975b5542a3" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 34937 } } peers { permanent_uuid: "897c2aff10254c7ebbb243fe7f37af67" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 33185 } } }
I20260806 11:41:16.422482 13102 sys_catalog.cc:458] T 00000000000000000000000000000000 P 897c2aff10254c7ebbb243fe7f37af67 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 11:41:16.423440 13116 mvcc.cc:205] Tried to move back new op lower bound from 7315523487312232448 to 7315523487178141696. Current Snapshot: MvccSnapshot[applied={T|T < 7315523487312232448}]
I20260806 11:41:16.424803 13101 catalog_manager.cc:1347] Generated new cluster ID: d06d6931d7b9469299cb460fb1af3cd3
I20260806 11:41:16.424881 13101 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 11:41:16.443068 13101 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 11:41:16.446681 13101 catalog_manager.cc:1504] Loading token signing keys...
I20260806 11:41:16.463013 13101 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b300ea3e113d4528911b80e89f04bfdd: Generated new TSK 0
I20260806 11:41:16.464093 13101 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 11:41:16.500103 13060 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:41:16.496802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37912 (local address 127.21.14.254:35671)
0806 11:41:16.496978 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:16.496983 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:16.498902 (+ 1919us) server_negotiation.cc:407] Connection header received
0806 11:41:16.499069 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:16.499072 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:16.499131 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:16.499230 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:16.499945 (+ 715us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37912: BlockingRecv error: recv got EOF from 127.0.0.1:37912 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260806 11:41:16.500367 13059 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:41:16.497543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49316 (local address 127.21.14.252:33185)
0806 11:41:16.497696 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:16.497700 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:16.499301 (+ 1601us) server_negotiation.cc:407] Connection header received
0806 11:41:16.499495 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:16.499498 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:16.499555 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:16.499626 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:16.500229 (+ 603us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49316: BlockingRecv error: recv got EOF from 127.0.0.1:49316 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260806 11:41:16.501834 12980 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:41:16.498259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45626 (local address 127.21.14.253:34937)
0806 11:41:16.498390 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:16.498394 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:16.500810 (+ 2416us) server_negotiation.cc:407] Connection header received
0806 11:41:16.500995 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:16.500998 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:16.501055 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:16.501110 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:16.501684 (+ 574us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45626: BlockingRecv error: recv got EOF from 127.0.0.1:45626 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260806 11:41:16.506829 12980 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:41:16.505122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45636 (local address 127.21.14.253:34937)
0806 11:41:16.505240 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:16.505245 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:16.505745 (+ 500us) server_negotiation.cc:407] Connection header received
0806 11:41:16.505984 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:16.505987 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:16.506077 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:16.506135 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:16.506713 (+ 578us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45636: BlockingRecv error: recv got EOF from 127.0.0.1:45636 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260806 11:41:16.512670 12980 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:41:16.510783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45648 (local address 127.21.14.253:34937)
0806 11:41:16.510998 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:16.511003 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:16.511634 (+ 631us) server_negotiation.cc:407] Connection header received
0806 11:41:16.511849 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:16.511853 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:16.511909 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:16.511963 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:16.512538 (+ 575us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45648: BlockingRecv error: recv got EOF from 127.0.0.1:45648 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260806 11:41:16.518656 12980 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:41:16.516862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45664 (local address 127.21.14.253:34937)
0806 11:41:16.516974 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:16.516978 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:16.517587 (+ 609us) server_negotiation.cc:407] Connection header received
0806 11:41:16.517813 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:16.517819 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:16.517875 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:16.517931 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:16.518527 (+ 596us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45664: BlockingRecv error: recv got EOF from 127.0.0.1:45664 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260806 11:41:16.527601 12980 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:41:16.525154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45680 (local address 127.21.14.253:34937)
0806 11:41:16.525333 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:16.525338 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:16.526016 (+ 678us) server_negotiation.cc:407] Connection header received
0806 11:41:16.526262 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:16.526266 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:16.526330 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:16.526394 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:16.527449 (+ 1055us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45680: BlockingRecv error: recv got EOF from 127.0.0.1:45680 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
W20260806 11:41:16.539973 12980 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:41:16.538158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45686 (local address 127.21.14.253:34937)
0806 11:41:16.538282 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:16.538286 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:16.538941 (+ 655us) server_negotiation.cc:407] Connection header received
0806 11:41:16.539155 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:16.539158 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:16.539214 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:16.539265 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:16.539846 (+ 581us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45686: BlockingRecv error: recv got EOF from 127.0.0.1:45686 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260806 11:41:16.560412 12980 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:41:16.558560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45690 (local address 127.21.14.253:34937)
0806 11:41:16.558698 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:16.558703 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:16.559367 (+ 664us) server_negotiation.cc:407] Connection header received
0806 11:41:16.559593 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:16.559596 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:16.559652 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:16.559706 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:16.560284 (+ 578us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45690: BlockingRecv error: recv got EOF from 127.0.0.1:45690 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260806 11:41:16.596731 12980 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:41:16.595206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45700 (local address 127.21.14.253:34937)
0806 11:41:16.595368 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:16.595383 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:16.595617 (+ 234us) server_negotiation.cc:407] Connection header received
0806 11:41:16.595813 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:16.595818 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:16.595883 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:16.595987 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:16.596582 (+ 595us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45700: BlockingRecv error: recv got EOF from 127.0.0.1:45700 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260806 11:41:16.664506 12980 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:41:16.663109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45706 (local address 127.21.14.253:34937)
0806 11:41:16.663339 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:16.663361 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:16.663481 (+ 120us) server_negotiation.cc:407] Connection header received
0806 11:41:16.663592 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:16.663594 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:16.663644 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:16.663720 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:16.664348 (+ 628us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45706: BlockingRecv error: recv got EOF from 127.0.0.1:45706 (error 108)
Metrics: {"server-negotiator.queue_time_us":124}
W20260806 11:41:16.796118 12980 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:41:16.794804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45716 (local address 127.21.14.253:34937)
0806 11:41:16.795001 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:16.795006 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:16.795158 (+ 152us) server_negotiation.cc:407] Connection header received
0806 11:41:16.795345 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:16.795352 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:16.795419 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:16.795502 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:16.795990 (+ 488us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45716: BlockingRecv error: recv got EOF from 127.0.0.1:45716 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
W20260806 11:41:17.055536 12980 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:41:17.054367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45724 (local address 127.21.14.253:34937)
0806 11:41:17.054506 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:17.054511 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:17.054615 (+ 104us) server_negotiation.cc:407] Connection header received
0806 11:41:17.054743 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:17.054746 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:17.054804 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:17.054937 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:17.055406 (+ 469us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45724: BlockingRecv error: recv got EOF from 127.0.0.1:45724 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 11:41:17.347905 13089 catalog_manager.cc:1259] Loaded cluster ID: d06d6931d7b9469299cb460fb1af3cd3
I20260806 11:41:17.348013 13089 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3e6a50bf43cd4cbcad84c8975b5542a3: loading cluster ID for follower catalog manager: success
I20260806 11:41:17.349886 13089 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3e6a50bf43cd4cbcad84c8975b5542a3: acquiring CA information for follower catalog manager: success
I20260806 11:41:17.351251 13089 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3e6a50bf43cd4cbcad84c8975b5542a3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 11:41:17.392418 13114 catalog_manager.cc:1259] Loaded cluster ID: d06d6931d7b9469299cb460fb1af3cd3
I20260806 11:41:17.392522 13114 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 897c2aff10254c7ebbb243fe7f37af67: loading cluster ID for follower catalog manager: success
I20260806 11:41:17.394354 13114 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 897c2aff10254c7ebbb243fe7f37af67: acquiring CA information for follower catalog manager: success
I20260806 11:41:17.395607 13114 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 897c2aff10254c7ebbb243fe7f37af67: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 11:41:17.574278 13191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:17.569992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45740 (local address 127.21.14.253:34937)
0806 11:41:17.570307 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:17.570312 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:17.570329 (+ 17us) server_negotiation.cc:407] Connection header received
0806 11:41:17.570485 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:17.570488 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:17.570675 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:17.570800 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 11:41:17.571125 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:17.571691 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:17.572635 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:17.572881 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:17.572996 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:41:17.573359 (+ 363us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 11:41:17.573821 (+ 462us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 11:41:17.573965 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 11:41:17.574029 (+ 64us) server_negotiation.cc:299] Negotiation successful
0806 11:41:17.574094 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":103,"threads_started":1}
I20260806 11:41:17.581943 13197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:17.577877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49328 (local address 127.21.14.252:33185)
0806 11:41:17.578165 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:17.578170 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:17.578186 (+ 16us) server_negotiation.cc:407] Connection header received
0806 11:41:17.578336 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:17.578339 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:17.578464 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:17.578601 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 11:41:17.578888 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:17.579415 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:17.580278 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:17.580530 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:17.580650 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:41:17.580986 (+ 336us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 11:41:17.581538 (+ 552us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 11:41:17.581675 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 11:41:17.581716 (+ 41us) server_negotiation.cc:299] Negotiation successful
0806 11:41:17.581771 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":93,"threads_started":1}
I20260806 11:41:17.590124 13191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:17.585424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45742 (local address 127.21.14.253:34937)
0806 11:41:17.585582 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:17.585587 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:17.585741 (+ 154us) server_negotiation.cc:407] Connection header received
0806 11:41:17.585972 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:17.585975 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:17.586037 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:17.586128 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 11:41:17.586464 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:17.587312 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:17.588377 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:17.588631 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:17.588736 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:41:17.589536 (+ 800us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 11:41:17.589621 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 11:41:17.589686 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 11:41:17.589843 (+ 157us) server_negotiation.cc:299] Negotiation successful
0806 11:41:17.589931 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 11:41:17.590286 13204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:17.585344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37918 (local address 127.21.14.254:35671)
0806 11:41:17.586004 (+ 660us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:17.586009 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:17.586022 (+ 13us) server_negotiation.cc:407] Connection header received
0806 11:41:17.586121 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:17.586124 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:17.586411 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:17.586570 (+ 159us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 11:41:17.586799 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:17.587570 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:17.588688 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:17.589042 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:17.589142 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:41:17.589432 (+ 290us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 11:41:17.589820 (+ 388us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 11:41:17.589911 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 11:41:17.590039 (+ 128us) server_negotiation.cc:299] Negotiation successful
0806 11:41:17.590122 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":580,"thread_start_us":72,"threads_started":1}
I20260806 11:41:17.590324 13197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:17.585854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49332 (local address 127.21.14.252:33185)
0806 11:41:17.586010 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:17.586014 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:17.586576 (+ 562us) server_negotiation.cc:407] Connection header received
0806 11:41:17.586736 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:17.586739 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:17.586792 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:17.586912 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 11:41:17.587155 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:17.587887 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:17.589008 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:17.589341 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:17.589444 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:41:17.589760 (+ 316us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 11:41:17.589832 (+ 72us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 11:41:17.589921 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 11:41:17.590106 (+ 185us) server_negotiation.cc:299] Negotiation successful
0806 11:41:17.590170 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 11:41:17.593837 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 12828
I20260806 11:41:17.603892 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 12905
I20260806 11:41:17.613065 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 12984
2026-08-06T11:41:17Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3225 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 11:41:17 dist-test-slave-7c35 krb5kdc[13208](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 11:41:17 dist-test-slave-7c35 krb5kdc[13208](info): set up 2 sockets
Aug 06 11:41:17 dist-test-slave-7c35 krb5kdc[13208](info): commencing operation
krb5kdc: starting...
W20260806 11:41:18.722365 21563 mini_kdc.cc:121] Time spent starting KDC: real 1.082s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/krb5kdc/test-user.keytab.
Aug 06 11:41:18 dist-test-slave-7c35 krb5kdc[13208](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016478, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T11:41:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T11:41:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.14.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab.
Entry for principal kudu/127.21.14.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab.
I20260806 11:41:18.861776 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:42233
--webserver_interface=127.21.14.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab
--principal=kudu/127.21.14.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:35681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.14.254:42233,127.21.14.253:42141,127.21.14.252:39035
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 11:41:18.981302 13221 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 11:41:18.981703 13221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:41:18.981783 13221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:41:18.981868 13221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:41:18.986199 13221 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 11:41:18.986320 13221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:41:18.986356 13221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:41:18.986399 13221 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 11:41:18.986436 13221 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 11:41:18.992276 13221 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:35681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.14.254:42233,127.21.14.253:42141,127.21.14.252:39035
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.254.keytab
--principal=kudu/127.21.14.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.254:42233
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.14.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13221
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:41:18.993549 13221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:41:18.994644 13221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:41:19.003847 13228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:41:19.003847 13231 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:41:19.003976 13221 server_base.cc:1034] running on GCE node
W20260806 11:41:19.003881 13227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:41:19.004733 13221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:41:19.005959 13221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:41:19.007126 13221 hybrid_clock.cc:648] HybridClock initialized: now 1786016479007111 us; error 31 us; skew 500 ppm
Aug 06 11:41:19 dist-test-slave-7c35 krb5kdc[13208](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016479, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:41:19.010164 13221 init.cc:377] Logged in from keytab as kudu/127.21.14.254@KRBTEST.COM (short username kudu)
I20260806 11:41:19.011296 13221 webserver.cc:460] Webserver started at http://127.21.14.254:38131/ using document root <none> and password file <none>
I20260806 11:41:19.011911 13221 fs_manager.cc:359] Metadata directory not provided
I20260806 11:41:19.011976 13221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:41:19.012212 13221 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:41:19.014436 13221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/minicluster-data/master-0/data/instance:
uuid: "2d125b8f73b043ccaa2e235c9a69297e"
format_stamp: "Formatted at 2026-08-06 11:41:19 on dist-test-slave-7c35"
server_key: "e011f32b155a06dd318cc0fcaeb67aef"
server_key_iv: "190ef1d8ccee44669bb689e145c7fa37"
server_key_version: "encryptionkey@0"
I20260806 11:41:19.015105 13221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/minicluster-data/master-0/wal/instance:
uuid: "2d125b8f73b043ccaa2e235c9a69297e"
format_stamp: "Formatted at 2026-08-06 11:41:19 on dist-test-slave-7c35"
server_key: "e011f32b155a06dd318cc0fcaeb67aef"
server_key_iv: "190ef1d8ccee44669bb689e145c7fa37"
server_key_version: "encryptionkey@0"
I20260806 11:41:19.019577 13221 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 11:41:19.022753 13239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:41:19.024122 13221 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 11:41:19.024271 13221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/minicluster-data/master-0/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/minicluster-data/master-0/wal
uuid: "2d125b8f73b043ccaa2e235c9a69297e"
format_stamp: "Formatted at 2026-08-06 11:41:19 on dist-test-slave-7c35"
server_key: "e011f32b155a06dd318cc0fcaeb67aef"
server_key_iv: "190ef1d8ccee44669bb689e145c7fa37"
server_key_version: "encryptionkey@0"
I20260806 11:41:19.024394 13221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:41:19.044848 13221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:41:19.048002 13221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:41:19.048225 13221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:41:19.056246 13221 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.254:42233
I20260806 11:41:19.056560 13292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.254:42233 every 8 connection(s)
I20260806 11:41:19.057417 13221 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/minicluster-data/master-0/data/info.pb
I20260806 11:41:19.060030 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 13221
I20260806 11:41:19.060161 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/minicluster-data/master-0/wal/instance
I20260806 11:41:19.060453 21563 external_mini_cluster.cc:1428] Setting key ca3bd9013f702cf71ba6ead6849c50c5
I20260806 11:41:19.060660 13293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:41:19.065220 13293 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.14.254" port: 42233 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.21.14.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.253@KRBTEST.COM" created.
Aug 06 11:41:19 dist-test-slave-7c35 krb5kdc[13208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016479, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.254@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:41:19.083034 13296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:19.068388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:42233 (local address 127.0.0.1:32946)
0806 11:41:19.068950 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0806 11:41:19.069006 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 11:41:19.069981 (+ 975us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:41:19.070951 (+ 970us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:41:19.070956 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:41:19.071394 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:41:19.071699 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:41:19.071714 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:41:19.073035 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:41:19.073038 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:41:19.073883 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:41:19.073895 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:41:19.074612 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:41:19.074617 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:41:19.074735 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:41:19.075544 (+ 809us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:41:19.075564 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:41:19.077529 (+ 1965us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:41:19.079678 (+ 2149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:41:19.079690 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:41:19.079702 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:41:19.079951 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:41:19.080295 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:41:19.080299 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:41:19.080301 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:41:19.080422 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:41:19.081102 (+ 680us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:41:19.081109 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:41:19.081265 (+ 156us) client_negotiation.cc:769] Sending connection context
0806 11:41:19.081447 (+ 182us) client_negotiation.cc:240] Negotiation successful
0806 11:41:19.081883 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"mutex_wait_us":122,"thread_start_us":185,"threads_started":1}
I20260806 11:41:19.083034 13297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:19.068388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32946 (local address 127.21.14.254:42233)
0806 11:41:19.069469 (+ 1081us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:19.069480 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:19.069524 (+ 44us) server_negotiation.cc:407] Connection header received
0806 11:41:19.070405 (+ 881us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:19.070415 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:19.070688 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:19.070941 (+ 253us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:19.071863 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:19.072879 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:19.074018 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:19.074429 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:19.074643 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:41:19.077696 (+ 3053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:41:19.077702 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:41:19.077716 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:41:19.077740 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:41:19.079544 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:41:19.080069 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:41:19.080083 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:41:19.080085 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:41:19.080193 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:41:19.080516 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:41:19.080521 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:41:19.080522 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:41:19.080975 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:41:19.081134 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 11:41:19.081514 (+ 380us) server_negotiation.cc:299] Negotiation successful
0806 11:41:19.081830 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":968,"thread_start_us":186,"threads_started":1}
I20260806 11:41:19.085893 13293 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.14.253" port: 42141 } has no permanent_uuid. Determining permanent_uuid...
I20260806 11:41:19.086995 13296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:19.086541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.253:42141 (local address 127.0.0.1:48108)
0806 11:41:19.086753 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0806 11:41:19.086897 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.253:42141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
W20260806 11:41:19.087280 13243 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.14.253:42141: connect: Connection refused (error 111)
I20260806 11:41:19.088474 13296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:19.088205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.253:42141 (local address 127.0.0.1:48118)
0806 11:41:19.088352 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 11:41:19.088404 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.253:42141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260806 11:41:19.088697 13293 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.14.253:42141: Network error: Client connection negotiation failed: client connection to 127.21.14.253:42141: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.253.keytab.
Entry for principal kudu/127.21.14.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.253.keytab.
I20260806 11:41:19.095906 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.253:42141
--webserver_interface=127.21.14.253
--webserver_port=0
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.253.keytab
--principal=kudu/127.21.14.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:35681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.14.254:42233,127.21.14.253:42141,127.21.14.252:39035
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 11:41:19.138046 13293 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.14.253" port: 42141 } attempt: 1
I20260806 11:41:19.139182 13296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:19.138770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.253:42141 (local address 127.0.0.1:48128)
0806 11:41:19.139023 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0806 11:41:19.139096 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.253:42141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138}
I20260806 11:41:19.140138 13296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:19.139847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.253:42141 (local address 127.0.0.1:48134)
0806 11:41:19.140012 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 11:41:19.140069 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.253:42141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260806 11:41:19.140345 13293 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.14.253:42141: Network error: Client connection negotiation failed: client connection to 127.21.14.253:42141: connect: Connection refused (error 111)
I20260806 11:41:19.208565 13293 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.14.253" port: 42141 } attempt: 2
I20260806 11:41:19.209492 13296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:19.209185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.253:42141 (local address 127.0.0.1:48136)
0806 11:41:19.209350 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 11:41:19.209413 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.253:42141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260806 11:41:19.210309 13296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:19.210096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.253:42141 (local address 127.0.0.1:48144)
0806 11:41:19.210203 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0806 11:41:19.210249 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.253:42141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260806 11:41:19.210486 13293 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.14.253:42141: Network error: Client connection negotiation failed: client connection to 127.21.14.253:42141: connect: Connection refused (error 111)
W20260806 11:41:19.221357 13301 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 11:41:19.221673 13301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:41:19.221698 13301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:41:19.221740 13301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:41:19.225915 13301 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 11:41:19.225993 13301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:41:19.226014 13301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:41:19.226032 13301 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 11:41:19.226048 13301 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 11:41:19.231914 13301 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:35681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.21.14.254:42233,127.21.14.253:42141,127.21.14.252:39035
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.253.keytab
--principal=kudu/127.21.14.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.253:42141
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.21.14.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13301
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:41:19.233094 13301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:41:19.234175 13301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 11:41:19.243319 13309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:41:19.243314 13306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:41:19.243306 13307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:41:19.243717 13301 server_base.cc:1034] running on GCE node
I20260806 11:41:19.244323 13301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:41:19.245517 13301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:41:19.246719 13301 hybrid_clock.cc:648] HybridClock initialized: now 1786016479246705 us; error 28 us; skew 500 ppm
Aug 06 11:41:19 dist-test-slave-7c35 krb5kdc[13208](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016479, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:41:19.249918 13301 init.cc:377] Logged in from keytab as kudu/127.21.14.253@KRBTEST.COM (short username kudu)
I20260806 11:41:19.251104 13301 webserver.cc:460] Webserver started at http://127.21.14.253:33303/ using document root <none> and password file <none>
I20260806 11:41:19.251751 13301 fs_manager.cc:359] Metadata directory not provided
I20260806 11:41:19.251853 13301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:41:19.252269 13301 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:41:19.254630 13301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/minicluster-data/master-1/data/instance:
uuid: "dd27c41696ba4c2d893b6a4296cb337e"
format_stamp: "Formatted at 2026-08-06 11:41:19 on dist-test-slave-7c35"
server_key: "1a4c234fb37e3ddc527953ed44984e4c"
server_key_iv: "cb4cd066f48514a75fb7359f5af129c8"
server_key_version: "encryptionkey@0"
I20260806 11:41:19.255347 13301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/minicluster-data/master-1/wal/instance:
uuid: "dd27c41696ba4c2d893b6a4296cb337e"
format_stamp: "Formatted at 2026-08-06 11:41:19 on dist-test-slave-7c35"
server_key: "1a4c234fb37e3ddc527953ed44984e4c"
server_key_iv: "cb4cd066f48514a75fb7359f5af129c8"
server_key_version: "encryptionkey@0"
I20260806 11:41:19.259665 13301 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 11:41:19.262428 13316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:41:19.263974 13301 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 11:41:19.264142 13301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/minicluster-data/master-1/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/minicluster-data/master-1/wal
uuid: "dd27c41696ba4c2d893b6a4296cb337e"
format_stamp: "Formatted at 2026-08-06 11:41:19 on dist-test-slave-7c35"
server_key: "1a4c234fb37e3ddc527953ed44984e4c"
server_key_iv: "cb4cd066f48514a75fb7359f5af129c8"
server_key_version: "encryptionkey@0"
I20260806 11:41:19.264274 13301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:41:19.283640 13301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:41:19.287818 13301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:41:19.288254 13301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:41:19.297833 13301 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.253:42141
I20260806 11:41:19.298188 13370 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.253:42141 every 8 connection(s)
I20260806 11:41:19.299093 13301 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/minicluster-data/master-1/data/info.pb
I20260806 11:41:19.301671 13293 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.14.253" port: 42141 } attempt: 3
I20260806 11:41:19.302690 13371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:41:19.304631 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 13301
I20260806 11:41:19.304801 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/minicluster-data/master-1/wal/instance
I20260806 11:41:19.305092 21563 external_mini_cluster.cc:1428] Setting key 30660965995417f6785379c76eb26466
I20260806 11:41:19.308264 13371 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.14.254" port: 42233 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 11:41:19 dist-test-slave-7c35 krb5kdc[13208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016479, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.254@KRBTEST.COM for kudu/127.21.14.253@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.14.252@KRBTEST.COM; defaulting to no policy
I20260806 11:41:19.318815 13296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:19.302258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.253:42141 (local address 127.0.0.1:48156)
0806 11:41:19.302412 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 11:41:19.302431 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 11:41:19.302540 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:41:19.305674 (+ 3134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:41:19.305680 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:41:19.305920 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:41:19.306121 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:41:19.306134 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:41:19.308106 (+ 1972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:41:19.308113 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:41:19.309056 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:41:19.309079 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:41:19.310155 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:41:19.310160 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:41:19.310396 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:41:19.311438 (+ 1042us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:41:19.311457 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:41:19.313236 (+ 1779us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:41:19.316306 (+ 3070us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:41:19.316310 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:41:19.316312 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:41:19.316635 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:41:19.317127 (+ 492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:41:19.317133 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:41:19.317136 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:41:19.317250 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:41:19.318113 (+ 863us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:41:19.318115 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:41:19.318219 (+ 104us) client_negotiation.cc:769] Sending connection context
0806 11:41:19.318322 (+ 103us) client_negotiation.cc:240] Negotiation successful
0806 11:41:19.318546 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260806 11:41:19.319922 13374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:19.303154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48156 (local address 127.21.14.253:42141)
0806 11:41:19.303632 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:19.303653 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:19.303698 (+ 45us) server_negotiation.cc:407] Connection header received
0806 11:41:19.304634 (+ 936us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:19.304660 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:19.305120 (+ 460us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:19.305707 (+ 587us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:19.306314 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:19.307910 (+ 1596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:19.309234 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:19.309953 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:19.310173 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:41:19.313546 (+ 3373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:41:19.313557 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:41:19.313573 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:41:19.313614 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:41:19.316154 (+ 2540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:41:19.316795 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:41:19.316804 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:41:19.316814 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:41:19.316983 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:41:19.317406 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:41:19.317411 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:41:19.317414 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:41:19.317984 (+ 570us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:41:19.318091 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 11:41:19.318604 (+ 513us) server_negotiation.cc:299] Negotiation successful
0806 11:41:19.318961 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":133,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.14.252@KRBTEST.COM" created.
Aug 06 11:41:19 dist-test-slave-7c35 krb5kdc[13208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016479, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.253@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:41:19.322587 13293 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.14.252" port: 39035 } has no permanent_uuid. Determining permanent_uuid...
I20260806 11:41:19.323798 13296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:19.323222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.252:39035 (local address 127.0.0.1:55568)
0806 11:41:19.323666 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0806 11:41:19.323720 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.252:39035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350}
I20260806 11:41:19.325310 13296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:19.324985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.252:39035 (local address 127.0.0.1:55570)
0806 11:41:19.325189 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0806 11:41:19.325247 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.252:39035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
W20260806 11:41:19.325503 13293 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.14.252:39035: Network error: Client connection negotiation failed: client connection to 127.21.14.252:39035: connect: Connection refused (error 111)
I20260806 11:41:19.326954 13297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:19.311089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32958 (local address 127.21.14.254:42233)
0806 11:41:19.311663 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:19.311670 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:19.312028 (+ 358us) server_negotiation.cc:407] Connection header received
0806 11:41:19.312279 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:19.312283 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:19.312353 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:19.312454 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:19.313491 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:19.314138 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:19.315331 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:19.316088 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:19.316242 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:41:19.322238 (+ 5996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:41:19.322243 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:41:19.322246 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:41:19.322279 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:41:19.324356 (+ 2077us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:41:19.324966 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:41:19.324976 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:41:19.324977 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:41:19.325087 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:41:19.325454 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:41:19.325460 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:41:19.325464 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:41:19.325746 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:41:19.325858 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 11:41:19.326619 (+ 761us) server_negotiation.cc:299] Negotiation successful
0806 11:41:19.326746 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431}
I20260806 11:41:19.327054 13376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:19.310597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:42233 (local address 127.0.0.1:32958)
0806 11:41:19.311830 (+ 1233us) negotiation.cc:107] Waiting for socket to connect
0806 11:41:19.311908 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0806 11:41:19.312076 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:41:19.312525 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:41:19.312534 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:41:19.313060 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:41:19.313292 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:41:19.313304 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:41:19.314323 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:41:19.314326 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:41:19.315187 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:41:19.315199 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:41:19.316259 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:41:19.316264 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:41:19.316416 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:41:19.317013 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:41:19.317027 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:41:19.322032 (+ 5005us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:41:19.324508 (+ 2476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:41:19.324520 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:41:19.324525 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:41:19.324799 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:41:19.325193 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:41:19.325199 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:41:19.325204 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:41:19.325327 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:41:19.326042 (+ 715us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:41:19.326059 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:41:19.326240 (+ 181us) client_negotiation.cc:769] Sending connection context
0806 11:41:19.326431 (+ 191us) client_negotiation.cc:240] Negotiation successful
0806 11:41:19.326704 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1031,"thread_start_us":244,"threads_started":1}
I20260806 11:41:19.328332 13371 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.14.253" port: 42141 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 11:41:19 dist-test-slave-7c35 krb5kdc[13208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016479, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.253@KRBTEST.COM for kudu/127.21.14.253@KRBTEST.COM
I20260806 11:41:19.340198 13376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:19.329000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.253:42141 (local address 127.0.0.1:48166)
0806 11:41:19.329108 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0806 11:41:19.329120 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 11:41:19.329195 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:41:19.329640 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:41:19.329643 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:41:19.329803 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:41:19.329910 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:41:19.329916 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:41:19.331061 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:41:19.331064 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:41:19.331628 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:41:19.331636 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:41:19.332335 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:41:19.332338 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:41:19.332434 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:41:19.332965 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:41:19.332977 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:41:19.335891 (+ 2914us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:41:19.338052 (+ 2161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:41:19.338055 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:41:19.338057 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:41:19.338269 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:41:19.338803 (+ 534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:41:19.338805 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:41:19.338807 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:41:19.338993 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:41:19.339553 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:41:19.339556 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:41:19.339647 (+ 91us) client_negotiation.cc:769] Sending connection context
0806 11:41:19.339795 (+ 148us) client_negotiation.cc:240] Negotiation successful
0806 11:41:19.339999 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260806 11:41:19.340198 13374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:19.329178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48166 (local address 127.21.14.253:42141)
0806 11:41:19.329361 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:19.329367 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:19.329385 (+ 18us) server_negotiation.cc:407] Connection header received
0806 11:41:19.329433 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:19.329438 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:19.329493 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:19.329592 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:19.330050 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:19.330933 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:19.331759 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:19.332185 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:19.332298 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:41:19.336092 (+ 3794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:41:19.336096 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:41:19.336098 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:41:19.336123 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:41:19.337904 (+ 1781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:41:19.338581 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:41:19.338584 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:41:19.338585 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:41:19.338674 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:41:19.339159 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:41:19.339166 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:41:19.339169 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:41:19.339453 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:41:19.339539 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 11:41:19.339939 (+ 400us) server_negotiation.cc:299] Negotiation successful
0806 11:41:19.340051 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260806 11:41:19.341130 13371 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.14.252" port: 39035 } has no permanent_uuid. Determining permanent_uuid...
I20260806 11:41:19.342037 13376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:19.341675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.252:39035 (local address 127.0.0.1:55586)
0806 11:41:19.341823 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 11:41:19.341939 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.252:39035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.14.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.252.keytab.
Entry for principal kudu/127.21.14.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.252.keytab.
W20260806 11:41:19.342768 13318 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.14.252:39035: connect: Connection refused (error 111)
I20260806 11:41:19.343384 21563 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4_zrTV/build/debug/bin/kudu
/tmp/dist-test-task4_zrTV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.252:39035
--webserver_interface=127.21.14.252
--webserver_port=0
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.252.keytab
--principal=kudu/127.21.14.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.14.212:35681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.14.254:42233,127.21.14.253:42141,127.21.14.252:39035
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 11:41:19.343837 13376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:19.343603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.252:39035 (local address 127.0.0.1:55588)
0806 11:41:19.343730 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 11:41:19.343778 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.252:39035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260806 11:41:19.344024 13371 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.14.252:39035: Network error: Client connection negotiation failed: client connection to 127.21.14.252:39035: connect: Connection refused (error 111)
I20260806 11:41:19.356703 13293 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.14.252" port: 39035 } attempt: 1
I20260806 11:41:19.357673 13296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:19.357351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.252:39035 (local address 127.0.0.1:55596)
0806 11:41:19.357517 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 11:41:19.357589 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.252:39035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260806 11:41:19.358471 13296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:19.358213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.252:39035 (local address 127.0.0.1:55598)
0806 11:41:19.358345 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 11:41:19.358409 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.252:39035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260806 11:41:19.358896 13293 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.14.252:39035: Network error: Client connection negotiation failed: client connection to 127.21.14.252:39035: connect: Connection refused (error 111)
I20260806 11:41:19.393289 13371 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.14.252" port: 39035 } attempt: 1
I20260806 11:41:19.394273 13376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:19.393960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.252:39035 (local address 127.0.0.1:55604)
0806 11:41:19.394129 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 11:41:19.394193 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.252:39035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260806 11:41:19.395077 13376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:19.394794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.252:39035 (local address 127.0.0.1:55620)
0806 11:41:19.394948 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 11:41:19.395001 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.252:39035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
W20260806 11:41:19.395236 13371 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.14.252:39035: Network error: Client connection negotiation failed: client connection to 127.21.14.252:39035: connect: Connection refused (error 111)
I20260806 11:41:19.434104 13293 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.14.252" port: 39035 } attempt: 2
I20260806 11:41:19.435132 13296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:19.434710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.252:39035 (local address 127.0.0.1:55626)
0806 11:41:19.434978 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0806 11:41:19.435046 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.252:39035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260806 11:41:19.435946 13296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:19.435713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.252:39035 (local address 127.0.0.1:55636)
0806 11:41:19.435830 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 11:41:19.435886 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.252:39035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260806 11:41:19.436095 13293 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.14.252:39035: Network error: Client connection negotiation failed: client connection to 127.21.14.252:39035: connect: Connection refused (error 111)
I20260806 11:41:19.463446 13371 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.14.252" port: 39035 } attempt: 2
I20260806 11:41:19.464411 13376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:19.464097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.252:39035 (local address 127.0.0.1:55640)
0806 11:41:19.464266 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 11:41:19.464330 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.252:39035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260806 11:41:19.465162 13376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:19.464941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.252:39035 (local address 127.0.0.1:55650)
0806 11:41:19.465052 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0806 11:41:19.465105 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.252:39035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260806 11:41:19.465296 13371 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.14.252:39035: Network error: Client connection negotiation failed: client connection to 127.21.14.252:39035: connect: Connection refused (error 111)
W20260806 11:41:19.466076 13379 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 11:41:19.466413 13379 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 11:41:19.466475 13379 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 11:41:19.466567 13379 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 11:41:19.471048 13379 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 11:41:19.471149 13379 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 11:41:19.471186 13379 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 11:41:19.471243 13379 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 11:41:19.471282 13379 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 11:41:19.477277 13379 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.14.212:35681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.21.14.254:42233,127.21.14.253:42141,127.21.14.252:39035
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/krb5kdc/kudu_127.21.14.252.keytab
--principal=kudu/127.21.14.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.14.252:39035
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.21.14.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13379
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 10:32:43 UTC on ccycloud.quasar-kjnemz.root.comops.site
build id 4570061
I20260806 11:41:19.478565 13379 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 11:41:19.479800 13379 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 11:41:19.489228 13379 server_base.cc:1034] running on GCE node
W20260806 11:41:19.489187 13387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:41:19.489181 13385 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 11:41:19.489181 13384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 11:41:19.490149 13379 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 11:41:19.491391 13379 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 11:41:19.492585 13379 hybrid_clock.cc:648] HybridClock initialized: now 1786016479492566 us; error 32 us; skew 500 ppm
Aug 06 11:41:19 dist-test-slave-7c35 krb5kdc[13208](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016479, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 11:41:19.496268 13379 init.cc:377] Logged in from keytab as kudu/127.21.14.252@KRBTEST.COM (short username kudu)
I20260806 11:41:19.497395 13379 webserver.cc:460] Webserver started at http://127.21.14.252:42351/ using document root <none> and password file <none>
I20260806 11:41:19.498035 13379 fs_manager.cc:359] Metadata directory not provided
I20260806 11:41:19.498104 13379 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 11:41:19.498342 13379 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 11:41:19.500660 13379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/minicluster-data/master-2/data/instance:
uuid: "f28ea935f3c842b6a754b86d3806ff38"
format_stamp: "Formatted at 2026-08-06 11:41:19 on dist-test-slave-7c35"
server_key: "ed333938eccd8ee2af3742c6368c697a"
server_key_iv: "8a0b5b33db5e2fad1d80381d67e816e3"
server_key_version: "encryptionkey@0"
I20260806 11:41:19.501286 13379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/minicluster-data/master-2/wal/instance:
uuid: "f28ea935f3c842b6a754b86d3806ff38"
format_stamp: "Formatted at 2026-08-06 11:41:19 on dist-test-slave-7c35"
server_key: "ed333938eccd8ee2af3742c6368c697a"
server_key_iv: "8a0b5b33db5e2fad1d80381d67e816e3"
server_key_version: "encryptionkey@0"
I20260806 11:41:19.505645 13379 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260806 11:41:19.508483 13395 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 11:41:19.510169 13379 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 11:41:19.510445 13379 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/minicluster-data/master-2/data,/tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/minicluster-data/master-2/wal
uuid: "f28ea935f3c842b6a754b86d3806ff38"
format_stamp: "Formatted at 2026-08-06 11:41:19 on dist-test-slave-7c35"
server_key: "ed333938eccd8ee2af3742c6368c697a"
server_key_iv: "8a0b5b33db5e2fad1d80381d67e816e3"
server_key_version: "encryptionkey@0"
I20260806 11:41:19.510583 13379 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 11:41:19.524892 13379 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 11:41:19.528693 13379 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 11:41:19.528916 13379 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 11:41:19.535245 13293 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.14.252" port: 39035 } attempt: 3
I20260806 11:41:19.536160 13296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:19.535799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.252:39035 (local address 127.0.0.1:55658)
0806 11:41:19.535975 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 11:41:19.536053 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.14.252:39035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260806 11:41:19.536801 13379 rpc_server.cc:307] RPC server started. Bound to: 127.21.14.252:39035
I20260806 11:41:19.537144 13448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.14.252:39035 every 8 connection(s)
I20260806 11:41:19.538007 13379 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/minicluster-data/master-2/data/info.pb
I20260806 11:41:19.540987 13450 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 11:41:19.541661 21563 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4_zrTV/build/debug/bin/kudu as pid 13379
I20260806 11:41:19.541797 21563 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4_zrTV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786016169045816-21563-0/minicluster-data/master-2/wal/instance
I20260806 11:41:19.542152 21563 external_mini_cluster.cc:1428] Setting key c7191312c6e7a4c8851d68ec1ca64350
Aug 06 11:41:19 dist-test-slave-7c35 krb5kdc[13208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016479, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.254@KRBTEST.COM for kudu/127.21.14.252@KRBTEST.COM
I20260806 11:41:19.546957 13450 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.14.254" port: 42233 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 11:41:19 dist-test-slave-7c35 krb5kdc[13208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016478, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.14.252@KRBTEST.COM
I20260806 11:41:19.551168 13296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:19.536820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.252:39035 (local address 127.0.0.1:55668)
0806 11:41:19.536958 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 11:41:19.536973 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 11:41:19.537072 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:41:19.540051 (+ 2979us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:41:19.540055 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:41:19.540263 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:41:19.540389 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:41:19.540398 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:41:19.542047 (+ 1649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:41:19.542051 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:41:19.542838 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:41:19.542896 (+ 58us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:41:19.543948 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:41:19.543952 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:41:19.544047 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:41:19.544632 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:41:19.544648 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:41:19.546101 (+ 1453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:41:19.548773 (+ 2672us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:41:19.548779 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:41:19.548783 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:41:19.549063 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:41:19.549502 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:41:19.549507 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:41:19.549510 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:41:19.549616 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:41:19.550470 (+ 854us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:41:19.550473 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:41:19.550599 (+ 126us) client_negotiation.cc:769] Sending connection context
0806 11:41:19.550744 (+ 145us) client_negotiation.cc:240] Negotiation successful
0806 11:41:19.550935 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260806 11:41:19.552595 13449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:19.538010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55668 (local address 127.21.14.252:39035)
0806 11:41:19.538466 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:19.538486 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:19.538555 (+ 69us) server_negotiation.cc:407] Connection header received
0806 11:41:19.539352 (+ 797us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:19.539377 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:19.539729 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:19.540126 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:19.540567 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:19.541831 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:19.543041 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:19.543797 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:19.543987 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:41:19.546315 (+ 2328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:41:19.546321 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:41:19.546335 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:41:19.546365 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:41:19.548649 (+ 2284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:41:19.549220 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:41:19.549227 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:41:19.549234 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:41:19.549404 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:41:19.549742 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:41:19.549746 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:41:19.549748 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:41:19.550366 (+ 618us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:41:19.550507 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 11:41:19.550940 (+ 433us) server_negotiation.cc:299] Negotiation successful
0806 11:41:19.551569 (+ 629us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":138,"threads_started":1}
I20260806 11:41:19.553987 13454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:19.542807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55678 (local address 127.21.14.252:39035)
0806 11:41:19.543314 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:19.543323 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:19.543339 (+ 16us) server_negotiation.cc:407] Connection header received
0806 11:41:19.543480 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:19.543486 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:19.543538 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:19.543638 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:19.544674 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:19.545207 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:19.546429 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:19.546796 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:19.546967 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:41:19.550111 (+ 3144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:41:19.550115 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:41:19.550117 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:41:19.550141 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:41:19.551995 (+ 1854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:41:19.552446 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:41:19.552450 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:41:19.552452 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:41:19.552541 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:41:19.552841 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:41:19.552845 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:41:19.552847 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:41:19.553061 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:41:19.553252 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0806 11:41:19.553471 (+ 219us) server_negotiation.cc:299] Negotiation successful
0806 11:41:19.553598 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":171,"threads_started":1}
I20260806 11:41:19.556468 13371 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.14.252" port: 39035 } attempt: 3
I20260806 11:41:19.557946 13293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d125b8f73b043ccaa2e235c9a69297e: Bootstrap starting.
Aug 06 11:41:19 dist-test-slave-7c35 krb5kdc[13208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016479, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.252@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
I20260806 11:41:19.561075 13293 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d125b8f73b043ccaa2e235c9a69297e: Neither blocks nor log segments found. Creating new log.
I20260806 11:41:19.562000 13293 log.cc:824] T 00000000000000000000000000000000 P 2d125b8f73b043ccaa2e235c9a69297e: Log is configured to *not* fsync() on all Append() calls
Aug 06 11:41:19 dist-test-slave-7c35 krb5kdc[13208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016478, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.14.254@KRBTEST.COM
Aug 06 11:41:19 dist-test-slave-7c35 krb5kdc[13208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016479, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.253@KRBTEST.COM for kudu/127.21.14.252@KRBTEST.COM
I20260806 11:41:19.565256 13293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d125b8f73b043ccaa2e235c9a69297e: No bootstrap required, opened a new log
I20260806 11:41:19.567513 13376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:19.556957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.252:39035 (local address 127.0.0.1:55682)
0806 11:41:19.557161 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0806 11:41:19.557176 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 11:41:19.557269 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:41:19.557517 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:41:19.557519 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:41:19.557704 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:41:19.557801 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:41:19.557807 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:41:19.558756 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:41:19.558760 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:41:19.559567 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:41:19.559580 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:41:19.560273 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:41:19.560277 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:41:19.560362 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:41:19.560849 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:41:19.560909 (+ 60us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:41:19.563022 (+ 2113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:41:19.565649 (+ 2627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:41:19.565653 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:41:19.565655 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:41:19.565928 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:41:19.566258 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:41:19.566261 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:41:19.566263 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:41:19.566347 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:41:19.566895 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:41:19.566898 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:41:19.567011 (+ 113us) client_negotiation.cc:769] Sending connection context
0806 11:41:19.567123 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 11:41:19.567294 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134}
I20260806 11:41:19.567668 13297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:19.549222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32970 (local address 127.21.14.254:42233)
0806 11:41:19.549357 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:19.549362 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:19.549742 (+ 380us) server_negotiation.cc:407] Connection header received
0806 11:41:19.549944 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:19.549947 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:19.549996 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:19.550049 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:19.551023 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:19.551785 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:19.552979 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:19.553331 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:19.553421 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:41:19.563602 (+ 10181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:41:19.563606 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:41:19.563610 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:41:19.563642 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:41:19.565488 (+ 1846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:41:19.566129 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:41:19.566133 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:41:19.566134 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:41:19.566228 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:41:19.566564 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:41:19.566567 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:41:19.566569 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:41:19.566791 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:41:19.566897 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 11:41:19.567381 (+ 484us) server_negotiation.cc:299] Negotiation successful
0806 11:41:19.567511 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 11:41:19.567871 13456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:19.555684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32972 (local address 127.21.14.254:42233)
0806 11:41:19.555961 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:19.555973 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:19.555984 (+ 11us) server_negotiation.cc:407] Connection header received
0806 11:41:19.556014 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:19.556016 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:19.556053 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:19.556125 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:19.556979 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:19.557861 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:19.558717 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:19.559101 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:19.559213 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:41:19.563022 (+ 3809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:41:19.563026 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:41:19.563028 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:41:19.563050 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:41:19.565709 (+ 2659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:41:19.566240 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:41:19.566244 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:41:19.566246 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:41:19.566337 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:41:19.566675 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:41:19.566678 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:41:19.566681 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:41:19.566954 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:41:19.567141 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0806 11:41:19.567547 (+ 406us) server_negotiation.cc:299] Negotiation successful
0806 11:41:19.567685 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":122,"threads_started":1}
I20260806 11:41:19.568019 13454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:19.557000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55682 (local address 127.21.14.252:39035)
0806 11:41:19.557111 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:19.557116 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:19.557252 (+ 136us) server_negotiation.cc:407] Connection header received
0806 11:41:19.557345 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:19.557347 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:19.557391 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:19.557474 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:19.557936 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:19.558613 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:19.559713 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:19.560052 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:19.560153 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:41:19.563178 (+ 3025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:41:19.563181 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:41:19.563184 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:41:19.563212 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:41:19.565520 (+ 2308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:41:19.566051 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:41:19.566055 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:41:19.566058 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:41:19.566147 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:41:19.566473 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:41:19.566476 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:41:19.566478 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:41:19.566727 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:41:19.566820 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 11:41:19.567743 (+ 923us) server_negotiation.cc:299] Negotiation successful
0806 11:41:19.567862 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 11:41:19.569383 13455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:19.549182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.254:42233 (local address 127.0.0.1:32970)
0806 11:41:19.549615 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0806 11:41:19.549667 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 11:41:19.549831 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:41:19.550220 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:41:19.550225 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:41:19.550684 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:41:19.550879 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:41:19.550888 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:41:19.551935 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:41:19.551938 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:41:19.552836 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:41:19.552847 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:41:19.553580 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:41:19.553584 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:41:19.553718 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:41:19.555009 (+ 1291us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:41:19.555027 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:41:19.563452 (+ 8425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:41:19.565622 (+ 2170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:41:19.565640 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:41:19.565648 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:41:19.565983 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:41:19.566354 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:41:19.566357 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:41:19.566358 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:41:19.566452 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:41:19.566994 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:41:19.567003 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:41:19.567142 (+ 139us) client_negotiation.cc:769] Sending connection context
0806 11:41:19.568965 (+ 1823us) client_negotiation.cc:240] Negotiation successful
0806 11:41:19.569197 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":121,"threads_started":1}
I20260806 11:41:19.570262 13293 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d125b8f73b043ccaa2e235c9a69297e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d125b8f73b043ccaa2e235c9a69297e" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 42233 } } peers { permanent_uuid: "dd27c41696ba4c2d893b6a4296cb337e" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 42141 } } peers { permanent_uuid: "f28ea935f3c842b6a754b86d3806ff38" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 39035 } }
I20260806 11:41:19.570670 13293 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d125b8f73b043ccaa2e235c9a69297e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d125b8f73b043ccaa2e235c9a69297e, State: Initialized, Role: FOLLOWER
I20260806 11:41:19.570822 13450 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.14.253" port: 42141 } has no permanent_uuid. Determining permanent_uuid...
I20260806 11:41:19.571574 13293 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d125b8f73b043ccaa2e235c9a69297e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d125b8f73b043ccaa2e235c9a69297e" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 42233 } } peers { permanent_uuid: "dd27c41696ba4c2d893b6a4296cb337e" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 42141 } } peers { permanent_uuid: "f28ea935f3c842b6a754b86d3806ff38" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 39035 } }
I20260806 11:41:19.571905 13371 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd27c41696ba4c2d893b6a4296cb337e: Bootstrap starting.
I20260806 11:41:19.572108 13459 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d125b8f73b043ccaa2e235c9a69297e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d125b8f73b043ccaa2e235c9a69297e" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 42233 } } peers { permanent_uuid: "dd27c41696ba4c2d893b6a4296cb337e" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 42141 } } peers { permanent_uuid: "f28ea935f3c842b6a754b86d3806ff38" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 39035 } } }
I20260806 11:41:19.572332 13459 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d125b8f73b043ccaa2e235c9a69297e [sys.catalog]: This master's current role is: FOLLOWER
I20260806 11:41:19.573107 13293 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d125b8f73b043ccaa2e235c9a69297e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 11:41:19.574601 13371 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dd27c41696ba4c2d893b6a4296cb337e: Neither blocks nor log segments found. Creating new log.
I20260806 11:41:19.575634 13371 log.cc:824] T 00000000000000000000000000000000 P dd27c41696ba4c2d893b6a4296cb337e: Log is configured to *not* fsync() on all Append() calls
Aug 06 11:41:19 dist-test-slave-7c35 krb5kdc[13208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016478, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.14.253@KRBTEST.COM
W20260806 11:41:19.579375 13471 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2d125b8f73b043ccaa2e235c9a69297e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 11:41:19.579504 13471 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 06 11:41:19 dist-test-slave-7c35 krb5kdc[13208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016479, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.252@KRBTEST.COM for kudu/127.21.14.253@KRBTEST.COM
I20260806 11:41:19.578817 13221 master_runner.cc:186] Error getting master registration for 127.21.14.253:42141: OK, instance_id { permanent_uuid: "dd27c41696ba4c2d893b6a4296cb337e" instance_seqno: 1786016479295332 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 11:41:19.582203 13221 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 11:41:19.581928 13374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:19.570042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48170 (local address 127.21.14.253:42141)
0806 11:41:19.570172 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:19.570176 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:19.570342 (+ 166us) server_negotiation.cc:407] Connection header received
0806 11:41:19.570533 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:19.570538 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:19.570587 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:19.570638 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:19.571939 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:19.572660 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:19.573744 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:19.574137 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:19.574236 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:41:19.577618 (+ 3382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:41:19.577622 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:41:19.577625 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:41:19.577652 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:41:19.579503 (+ 1851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:41:19.580310 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:41:19.580315 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:41:19.580317 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:41:19.580399 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:41:19.580742 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:41:19.580747 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:41:19.580750 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:41:19.581057 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:41:19.581224 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0806 11:41:19.581625 (+ 401us) server_negotiation.cc:299] Negotiation successful
0806 11:41:19.581750 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 11:41:19.585709 13455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:19.571373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.253:42141 (local address 127.0.0.1:48176)
0806 11:41:19.571485 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0806 11:41:19.571501 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 11:41:19.571579 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:41:19.576243 (+ 4664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:41:19.576248 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:41:19.576459 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:41:19.576623 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:41:19.576635 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:41:19.577798 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:41:19.577804 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:41:19.578794 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:41:19.578810 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:41:19.579656 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:41:19.579660 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:41:19.579769 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:41:19.580245 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:41:19.580264 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:41:19.581806 (+ 1542us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:41:19.583991 (+ 2185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:41:19.583994 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:41:19.583996 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:41:19.584247 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:41:19.584557 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:41:19.584561 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:41:19.584564 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:41:19.584659 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:41:19.585137 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:41:19.585141 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:41:19.585247 (+ 106us) client_negotiation.cc:769] Sending connection context
0806 11:41:19.585376 (+ 129us) client_negotiation.cc:240] Negotiation successful
0806 11:41:19.585542 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260806 11:41:19.585726 13458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:19.571439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48176 (local address 127.21.14.253:42141)
0806 11:41:19.575938 (+ 4499us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:19.575949 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:19.575967 (+ 18us) server_negotiation.cc:407] Connection header received
0806 11:41:19.576011 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:19.576014 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:19.576062 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:19.576767 (+ 705us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:19.576871 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:19.577667 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:19.578992 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:19.579396 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:19.580010 (+ 614us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:41:19.581970 (+ 1960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:41:19.581975 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:41:19.581977 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:41:19.582002 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:41:19.583857 (+ 1855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:41:19.584378 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:41:19.584380 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:41:19.584381 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:41:19.584452 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:41:19.584766 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:41:19.584770 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:41:19.584772 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:41:19.585005 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:41:19.585126 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 11:41:19.585436 (+ 310us) server_negotiation.cc:299] Negotiation successful
0806 11:41:19.585551 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4405,"thread_start_us":147,"threads_started":1}
I20260806 11:41:19.586114 21563 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 11:41:19.586627 13450 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.14.252" port: 39035 } has no permanent_uuid. Determining permanent_uuid...
I20260806 11:41:19.587428 13266 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "2d125b8f73b043ccaa2e235c9a69297e"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32972
I20260806 11:41:19.587692 13266 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d125b8f73b043ccaa2e235c9a69297e [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 11:41:19.587813 13266 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d125b8f73b043ccaa2e235c9a69297e [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:41:19.589864 13266 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d125b8f73b043ccaa2e235c9a69297e [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d125b8f73b043ccaa2e235c9a69297e" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 42233 } } peers { permanent_uuid: "dd27c41696ba4c2d893b6a4296cb337e" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 42141 } } peers { permanent_uuid: "f28ea935f3c842b6a754b86d3806ff38" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 39035 } }
Aug 06 11:41:19 dist-test-slave-7c35 krb5kdc[13208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786016479, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.14.252@KRBTEST.COM for kudu/127.21.14.252@KRBTEST.COM
I20260806 11:41:19.591966 13266 leader_election.cc:288] T 00000000000000000000000000000000 P 2d125b8f73b043ccaa2e235c9a69297e [CANDIDATE]: Term 1 election: Requested vote from peers dd27c41696ba4c2d893b6a4296cb337e (127.21.14.253:42141), f28ea935f3c842b6a754b86d3806ff38 (127.21.14.252:39035)
I20260806 11:41:19.595854 13455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:19.587145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.14.252:39035 (local address 127.0.0.1:55698)
0806 11:41:19.587312 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 11:41:19.587325 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 11:41:19.587408 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 11:41:19.587626 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 11:41:19.587628 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 11:41:19.587798 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0806 11:41:19.587862 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:41:19.587867 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:41:19.588591 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:41:19.588594 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:41:19.589138 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 11:41:19.589145 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 11:41:19.589594 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 11:41:19.589597 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 11:41:19.589665 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:41:19.590184 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 11:41:19.590206 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 11:41:19.591563 (+ 1357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 11:41:19.593725 (+ 2162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:41:19.593732 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:41:19.593737 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:41:19.593982 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:41:19.594464 (+ 482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 11:41:19.594469 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 11:41:19.594471 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 11:41:19.594602 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 11:41:19.595346 (+ 744us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 11:41:19.595349 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 11:41:19.595439 (+ 90us) client_negotiation.cc:769] Sending connection context
0806 11:41:19.595523 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 11:41:19.595674 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96}
I20260806 11:41:19.596505 13454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:19.587184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55698 (local address 127.21.14.252:39035)
0806 11:41:19.587290 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:19.587294 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:19.587370 (+ 76us) server_negotiation.cc:407] Connection header received
0806 11:41:19.587477 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:19.587482 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:19.587518 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:19.587614 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:19.587974 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:19.588469 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:19.589257 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:19.589480 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:19.589594 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:41:19.591714 (+ 2120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:41:19.591722 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:41:19.591724 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:41:19.591745 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:41:19.593584 (+ 1839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:41:19.594122 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:41:19.594126 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:41:19.594128 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:41:19.594315 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:41:19.594733 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:41:19.594736 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:41:19.594738 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:41:19.595203 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:41:19.596124 (+ 921us) server_negotiation.cc:1035] Waiting for connection context
0806 11:41:19.596223 (+ 99us) server_negotiation.cc:299] Negotiation successful
0806 11:41:19.596355 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 11:41:19.598477 13371 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd27c41696ba4c2d893b6a4296cb337e: No bootstrap required, opened a new log
I20260806 11:41:19.599988 13450 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f28ea935f3c842b6a754b86d3806ff38: Bootstrap starting.
I20260806 11:41:19.602947 13371 raft_consensus.cc:348] T 00000000000000000000000000000000 P dd27c41696ba4c2d893b6a4296cb337e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d125b8f73b043ccaa2e235c9a69297e" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 42233 } } peers { permanent_uuid: "dd27c41696ba4c2d893b6a4296cb337e" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 42141 } } peers { permanent_uuid: "f28ea935f3c842b6a754b86d3806ff38" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 39035 } }
I20260806 11:41:19.603238 13450 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f28ea935f3c842b6a754b86d3806ff38: Neither blocks nor log segments found. Creating new log.
I20260806 11:41:19.603348 13371 raft_consensus.cc:729] T 00000000000000000000000000000000 P dd27c41696ba4c2d893b6a4296cb337e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd27c41696ba4c2d893b6a4296cb337e, State: Initialized, Role: FOLLOWER
I20260806 11:41:19.603801 13371 consensus_queue.cc:260] T 00000000000000000000000000000000 P dd27c41696ba4c2d893b6a4296cb337e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d125b8f73b043ccaa2e235c9a69297e" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 42233 } } peers { permanent_uuid: "dd27c41696ba4c2d893b6a4296cb337e" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 42141 } } peers { permanent_uuid: "f28ea935f3c842b6a754b86d3806ff38" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 39035 } }
I20260806 11:41:19.604261 13450 log.cc:824] T 00000000000000000000000000000000 P f28ea935f3c842b6a754b86d3806ff38: Log is configured to *not* fsync() on all Append() calls
I20260806 11:41:19.604642 13480 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd27c41696ba4c2d893b6a4296cb337e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d125b8f73b043ccaa2e235c9a69297e" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 42233 } } peers { permanent_uuid: "dd27c41696ba4c2d893b6a4296cb337e" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 42141 } } peers { permanent_uuid: "f28ea935f3c842b6a754b86d3806ff38" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 39035 } } }
I20260806 11:41:19.604952 13480 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd27c41696ba4c2d893b6a4296cb337e [sys.catalog]: This master's current role is: FOLLOWER
I20260806 11:41:19.605490 13371 sys_catalog.cc:564] T 00000000000000000000000000000000 P dd27c41696ba4c2d893b6a4296cb337e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 11:41:19.606894 13343 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2d125b8f73b043ccaa2e235c9a69297e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "dd27c41696ba4c2d893b6a4296cb337e"
I20260806 11:41:19.607149 13343 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dd27c41696ba4c2d893b6a4296cb337e [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:41:19.607666 13450 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f28ea935f3c842b6a754b86d3806ff38: No bootstrap required, opened a new log
I20260806 11:41:19.608538 13343 raft_consensus.cc:2454] T 00000000000000000000000000000000 P dd27c41696ba4c2d893b6a4296cb337e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d125b8f73b043ccaa2e235c9a69297e in term 1.
I20260806 11:41:19.609112 13243 leader_election.cc:302] T 00000000000000000000000000000000 P 2d125b8f73b043ccaa2e235c9a69297e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d125b8f73b043ccaa2e235c9a69297e, dd27c41696ba4c2d893b6a4296cb337e; no voters:
I20260806 11:41:19.609444 13459 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d125b8f73b043ccaa2e235c9a69297e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 11:41:19.609684 13459 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d125b8f73b043ccaa2e235c9a69297e [term 1 LEADER]: Becoming Leader. State: Replica: 2d125b8f73b043ccaa2e235c9a69297e, State: Running, Role: LEADER
I20260806 11:41:19.610076 13459 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d125b8f73b043ccaa2e235c9a69297e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d125b8f73b043ccaa2e235c9a69297e" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 42233 } } peers { permanent_uuid: "dd27c41696ba4c2d893b6a4296cb337e" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 42141 } } peers { permanent_uuid: "f28ea935f3c842b6a754b86d3806ff38" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 39035 } }
I20260806 11:41:19.612488 13450 raft_consensus.cc:348] T 00000000000000000000000000000000 P f28ea935f3c842b6a754b86d3806ff38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d125b8f73b043ccaa2e235c9a69297e" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 42233 } } peers { permanent_uuid: "dd27c41696ba4c2d893b6a4296cb337e" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 42141 } } peers { permanent_uuid: "f28ea935f3c842b6a754b86d3806ff38" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 39035 } }
I20260806 11:41:19.612795 13450 raft_consensus.cc:729] T 00000000000000000000000000000000 P f28ea935f3c842b6a754b86d3806ff38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f28ea935f3c842b6a754b86d3806ff38, State: Initialized, Role: FOLLOWER
I20260806 11:41:19.613181 13450 consensus_queue.cc:260] T 00000000000000000000000000000000 P f28ea935f3c842b6a754b86d3806ff38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d125b8f73b043ccaa2e235c9a69297e" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 42233 } } peers { permanent_uuid: "dd27c41696ba4c2d893b6a4296cb337e" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 42141 } } peers { permanent_uuid: "f28ea935f3c842b6a754b86d3806ff38" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 39035 } }
I20260806 11:41:19.615016 13450 sys_catalog.cc:564] T 00000000000000000000000000000000 P f28ea935f3c842b6a754b86d3806ff38 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 11:41:19.618404 13493 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d125b8f73b043ccaa2e235c9a69297e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d125b8f73b043ccaa2e235c9a69297e. Latest consensus state: current_term: 1 leader_uuid: "2d125b8f73b043ccaa2e235c9a69297e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d125b8f73b043ccaa2e235c9a69297e" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 42233 } } peers { permanent_uuid: "dd27c41696ba4c2d893b6a4296cb337e" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 42141 } } peers { permanent_uuid: "f28ea935f3c842b6a754b86d3806ff38" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 39035 } } }
I20260806 11:41:19.618551 13493 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d125b8f73b043ccaa2e235c9a69297e [sys.catalog]: This master's current role is: LEADER
I20260806 11:41:19.621593 13456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:19.595248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32976 (local address 127.21.14.254:42233)
0806 11:41:19.595381 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:19.595386 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:19.595401 (+ 15us) server_negotiation.cc:407] Connection header received
0806 11:41:19.595646 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:19.595651 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:19.595708 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:19.595770 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:19.605517 (+ 9747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:19.606381 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:19.612460 (+ 6079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:19.612838 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:19.612985 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:41:19.615422 (+ 2437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:41:19.615427 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:41:19.615431 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:41:19.615467 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:41:19.617495 (+ 2028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:41:19.618144 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:41:19.618148 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:41:19.618149 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:41:19.618253 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:41:19.618604 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:41:19.618608 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:41:19.618609 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:41:19.618903 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:41:19.619030 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 11:41:19.621146 (+ 2116us) server_negotiation.cc:299] Negotiation successful
0806 11:41:19.621348 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 11:41:19.621639 13422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2d125b8f73b043ccaa2e235c9a69297e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f28ea935f3c842b6a754b86d3806ff38"
I20260806 11:41:19.624025 13422 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f28ea935f3c842b6a754b86d3806ff38 [term 0 FOLLOWER]: Advancing to term 1
I20260806 11:41:19.625586 13422 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f28ea935f3c842b6a754b86d3806ff38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d125b8f73b043ccaa2e235c9a69297e in term 1.
I20260806 11:41:19.631620 13454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:19.612605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55704 (local address 127.21.14.252:39035)
0806 11:41:19.612725 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:19.612730 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:19.619865 (+ 7135us) server_negotiation.cc:407] Connection header received
0806 11:41:19.620052 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:19.620055 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:19.620109 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:19.620162 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:19.622885 (+ 2723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:19.623626 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:19.624810 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:19.625126 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:19.625212 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:41:19.627256 (+ 2044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:41:19.627261 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:41:19.627263 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:41:19.627294 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:41:19.629160 (+ 1866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:41:19.629834 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:41:19.629838 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:41:19.629840 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:41:19.629946 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:41:19.630416 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:41:19.630420 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:41:19.630421 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:41:19.630711 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:41:19.630842 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 11:41:19.631285 (+ 443us) server_negotiation.cc:299] Negotiation successful
0806 11:41:19.631430 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 11:41:19.633337 13507 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 11:41:19.634593 13507 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260806 11:41:19.622339 13506 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f28ea935f3c842b6a754b86d3806ff38: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 11:41:19.634925 13506 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 11:41:19.622668 13495 sys_catalog.cc:455] T 00000000000000000000000000000000 P f28ea935f3c842b6a754b86d3806ff38 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d125b8f73b043ccaa2e235c9a69297e" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 42233 } } peers { permanent_uuid: "dd27c41696ba4c2d893b6a4296cb337e" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 42141 } } peers { permanent_uuid: "f28ea935f3c842b6a754b86d3806ff38" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 39035 } } }
I20260806 11:41:19.635090 13495 sys_catalog.cc:458] T 00000000000000000000000000000000 P f28ea935f3c842b6a754b86d3806ff38 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 11:41:19.642688 13458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:19.595720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48192 (local address 127.21.14.253:42141)
0806 11:41:19.596318 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:19.596324 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:19.612734 (+ 16410us) server_negotiation.cc:407] Connection header received
0806 11:41:19.613658 (+ 924us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:19.613661 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:19.613729 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:19.613791 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:19.632751 (+ 18960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:19.633702 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:19.635151 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:19.635529 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:19.635620 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:41:19.638596 (+ 2976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 11:41:19.638601 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 11:41:19.638604 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 11:41:19.638636 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 11:41:19.640608 (+ 1972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:41:19.641178 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:41:19.641182 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:41:19.641184 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:41:19.641280 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 11:41:19.641603 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 11:41:19.641607 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 11:41:19.641609 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 11:41:19.641847 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 11:41:19.642000 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0806 11:41:19.642367 (+ 367us) server_negotiation.cc:299] Negotiation successful
0806 11:41:19.642512 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150}
W20260806 11:41:19.643646 13492 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dd27c41696ba4c2d893b6a4296cb337e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 11:41:19.643798 13492 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 11:41:19.651329 13343 raft_consensus.cc:1261] T 00000000000000000000000000000000 P dd27c41696ba4c2d893b6a4296cb337e [term 1 FOLLOWER]: Refusing update from remote peer 2d125b8f73b043ccaa2e235c9a69297e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 11:41:19.652175 13459 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2d125b8f73b043ccaa2e235c9a69297e [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd27c41696ba4c2d893b6a4296cb337e" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 42141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 11:41:19.653636 13421 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f28ea935f3c842b6a754b86d3806ff38 [term 1 FOLLOWER]: Refusing update from remote peer 2d125b8f73b043ccaa2e235c9a69297e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 11:41:19.656049 13459 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2d125b8f73b043ccaa2e235c9a69297e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f28ea935f3c842b6a754b86d3806ff38" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 39035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 11:41:19.662338 13480 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd27c41696ba4c2d893b6a4296cb337e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d125b8f73b043ccaa2e235c9a69297e. Latest consensus state: current_term: 1 leader_uuid: "2d125b8f73b043ccaa2e235c9a69297e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d125b8f73b043ccaa2e235c9a69297e" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 42233 } } peers { permanent_uuid: "dd27c41696ba4c2d893b6a4296cb337e" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 42141 } } peers { permanent_uuid: "f28ea935f3c842b6a754b86d3806ff38" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 39035 } } }
I20260806 11:41:19.662509 13480 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd27c41696ba4c2d893b6a4296cb337e [sys.catalog]: This master's current role is: FOLLOWER
I20260806 11:41:19.671006 13495 sys_catalog.cc:455] T 00000000000000000000000000000000 P f28ea935f3c842b6a754b86d3806ff38 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d125b8f73b043ccaa2e235c9a69297e. Latest consensus state: current_term: 1 leader_uuid: "2d125b8f73b043ccaa2e235c9a69297e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d125b8f73b043ccaa2e235c9a69297e" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 42233 } } peers { permanent_uuid: "dd27c41696ba4c2d893b6a4296cb337e" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 42141 } } peers { permanent_uuid: "f28ea935f3c842b6a754b86d3806ff38" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 39035 } } }
I20260806 11:41:19.671195 13495 sys_catalog.cc:458] T 00000000000000000000000000000000 P f28ea935f3c842b6a754b86d3806ff38 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 11:41:19.675980 13459 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d125b8f73b043ccaa2e235c9a69297e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2d125b8f73b043ccaa2e235c9a69297e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d125b8f73b043ccaa2e235c9a69297e" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 42233 } } peers { permanent_uuid: "dd27c41696ba4c2d893b6a4296cb337e" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 42141 } } peers { permanent_uuid: "f28ea935f3c842b6a754b86d3806ff38" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 39035 } } }
I20260806 11:41:19.676146 13459 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d125b8f73b043ccaa2e235c9a69297e [sys.catalog]: This master's current role is: LEADER
I20260806 11:41:19.677716 13480 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd27c41696ba4c2d893b6a4296cb337e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2d125b8f73b043ccaa2e235c9a69297e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d125b8f73b043ccaa2e235c9a69297e" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 42233 } } peers { permanent_uuid: "dd27c41696ba4c2d893b6a4296cb337e" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 42141 } } peers { permanent_uuid: "f28ea935f3c842b6a754b86d3806ff38" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 39035 } } }
I20260806 11:41:19.677855 13480 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd27c41696ba4c2d893b6a4296cb337e [sys.catalog]: This master's current role is: FOLLOWER
I20260806 11:41:19.678719 13510 mvcc.cc:205] Tried to move back new op lower bound from 7315523500642082816 to 7315523500487155712. Current Snapshot: MvccSnapshot[applied={T|T < 7315523500642082816}]
I20260806 11:41:19.679237 13513 mvcc.cc:205] Tried to move back new op lower bound from 7315523500642082816 to 7315523500487155712. Current Snapshot: MvccSnapshot[applied={T|T < 7315523500642082816}]
I20260806 11:41:19.684289 13493 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d125b8f73b043ccaa2e235c9a69297e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2d125b8f73b043ccaa2e235c9a69297e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d125b8f73b043ccaa2e235c9a69297e" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 42233 } } peers { permanent_uuid: "dd27c41696ba4c2d893b6a4296cb337e" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 42141 } } peers { permanent_uuid: "f28ea935f3c842b6a754b86d3806ff38" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 39035 } } }
I20260806 11:41:19.684440 13493 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d125b8f73b043ccaa2e235c9a69297e [sys.catalog]: This master's current role is: LEADER
I20260806 11:41:19.686511 13495 sys_catalog.cc:455] T 00000000000000000000000000000000 P f28ea935f3c842b6a754b86d3806ff38 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2d125b8f73b043ccaa2e235c9a69297e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d125b8f73b043ccaa2e235c9a69297e" member_type: VOTER last_known_addr { host: "127.21.14.254" port: 42233 } } peers { permanent_uuid: "dd27c41696ba4c2d893b6a4296cb337e" member_type: VOTER last_known_addr { host: "127.21.14.253" port: 42141 } } peers { permanent_uuid: "f28ea935f3c842b6a754b86d3806ff38" member_type: VOTER last_known_addr { host: "127.21.14.252" port: 39035 } } }
I20260806 11:41:19.686652 13495 sys_catalog.cc:458] T 00000000000000000000000000000000 P f28ea935f3c842b6a754b86d3806ff38 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 11:41:19.687332 13515 mvcc.cc:205] Tried to move back new op lower bound from 7315523500642082816 to 7315523500487155712. Current Snapshot: MvccSnapshot[applied={T|T < 7315523500642082816}]
I20260806 11:41:19.688197 13507 catalog_manager.cc:1347] Generated new cluster ID: d69cf8610b6543e9a3a58f2eabc91d01
I20260806 11:41:19.688261 13507 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 11:41:19.730633 13507 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 11:41:19.732035 13507 catalog_manager.cc:1504] Loading token signing keys...
I20260806 11:41:19.742290 13507 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2d125b8f73b043ccaa2e235c9a69297e: Generated new TSK 0
I20260806 11:41:19.743309 13507 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 11:41:19.785844 13458 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:41:19.782646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48206 (local address 127.21.14.253:42141)
0806 11:41:19.782777 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:19.782782 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:19.784666 (+ 1884us) server_negotiation.cc:407] Connection header received
0806 11:41:19.784875 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:19.784878 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:19.784934 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:19.784986 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:19.785692 (+ 706us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48206: BlockingRecv error: recv got EOF from 127.0.0.1:48206 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260806 11:41:19.787387 13454 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:41:19.783432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55716 (local address 127.21.14.252:39035)
0806 11:41:19.783560 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:19.783564 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:19.786403 (+ 2839us) server_negotiation.cc:407] Connection header received
0806 11:41:19.786585 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:19.786588 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:19.786639 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:19.786690 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:19.787255 (+ 565us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55716: BlockingRecv error: recv got EOF from 127.0.0.1:55716 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260806 11:41:19.788676 13456 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:41:19.784130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32984 (local address 127.21.14.254:42233)
0806 11:41:19.784243 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:19.784249 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:19.787876 (+ 3627us) server_negotiation.cc:407] Connection header received
0806 11:41:19.787992 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:19.787996 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:19.788059 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:19.788143 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:19.788555 (+ 412us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32984: BlockingRecv error: recv got EOF from 127.0.0.1:32984 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260806 11:41:19.793551 13458 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:41:19.791796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48216 (local address 127.21.14.253:42141)
0806 11:41:19.791910 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:19.791915 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:19.792499 (+ 584us) server_negotiation.cc:407] Connection header received
0806 11:41:19.792728 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:19.792731 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:19.792815 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:19.792871 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:19.793435 (+ 564us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48216: BlockingRecv error: recv got EOF from 127.0.0.1:48216 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260806 11:41:19.798981 13458 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:41:19.797141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48218 (local address 127.21.14.253:42141)
0806 11:41:19.797270 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:19.797275 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:19.797880 (+ 605us) server_negotiation.cc:407] Connection header received
0806 11:41:19.798109 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:19.798112 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:19.798174 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:19.798231 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:19.798805 (+ 574us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48218: BlockingRecv error: recv got EOF from 127.0.0.1:48218 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260806 11:41:19.809116 13458 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:41:19.807241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48228 (local address 127.21.14.253:42141)
0806 11:41:19.807378 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:19.807382 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:19.807995 (+ 613us) server_negotiation.cc:407] Connection header received
0806 11:41:19.808243 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:19.808246 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:19.808307 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:19.808367 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:19.808970 (+ 603us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48228: BlockingRecv error: recv got EOF from 127.0.0.1:48228 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260806 11:41:19.818126 13458 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:41:19.816219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48232 (local address 127.21.14.253:42141)
0806 11:41:19.816445 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:19.816451 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:19.817043 (+ 592us) server_negotiation.cc:407] Connection header received
0806 11:41:19.817212 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:19.817215 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:19.817285 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:19.817407 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:19.817984 (+ 577us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48232: BlockingRecv error: recv got EOF from 127.0.0.1:48232 (error 108)
Metrics: {"server-negotiator.queue_time_us":137}
W20260806 11:41:19.830349 13458 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:41:19.829251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48242 (local address 127.21.14.253:42141)
0806 11:41:19.829401 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:19.829406 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:19.829422 (+ 16us) server_negotiation.cc:407] Connection header received
0806 11:41:19.829483 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:19.829486 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:19.829545 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:19.829639 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:19.830222 (+ 583us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48242: BlockingRecv error: recv got EOF from 127.0.0.1:48242 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260806 11:41:19.850241 13458 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:41:19.848875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48254 (local address 127.21.14.253:42141)
0806 11:41:19.849004 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:19.849009 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:19.849276 (+ 267us) server_negotiation.cc:407] Connection header received
0806 11:41:19.849427 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:19.849431 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:19.849490 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:19.849593 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:19.850108 (+ 515us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48254: BlockingRecv error: recv got EOF from 127.0.0.1:48254 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260806 11:41:19.886008 13458 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:41:19.884678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48260 (local address 127.21.14.253:42141)
0806 11:41:19.884885 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:19.884890 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:19.885009 (+ 119us) server_negotiation.cc:407] Connection header received
0806 11:41:19.885220 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:19.885224 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:19.885304 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:19.885402 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:19.885890 (+ 488us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48260: BlockingRecv error: recv got EOF from 127.0.0.1:48260 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
W20260806 11:41:19.953758 13458 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:41:19.952473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48266 (local address 127.21.14.253:42141)
0806 11:41:19.952656 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:19.952661 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:19.952778 (+ 117us) server_negotiation.cc:407] Connection header received
0806 11:41:19.952968 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:19.952973 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:19.953029 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:19.953144 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:19.953635 (+ 491us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48266: BlockingRecv error: recv got EOF from 127.0.0.1:48266 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
W20260806 11:41:20.086055 13458 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:41:20.084676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48276 (local address 127.21.14.253:42141)
0806 11:41:20.084860 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:20.084867 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:20.084884 (+ 17us) server_negotiation.cc:407] Connection header received
0806 11:41:20.085067 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:20.085071 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:20.085129 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:20.085245 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:20.085890 (+ 645us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48276: BlockingRecv error: recv got EOF from 127.0.0.1:48276 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
W20260806 11:41:20.346134 13458 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 11:41:20.344758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48284 (local address 127.21.14.253:42141)
0806 11:41:20.344939 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:20.344945 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:20.345063 (+ 118us) server_negotiation.cc:407] Connection header received
0806 11:41:20.345254 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:20.345257 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:20.345327 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:20.345426 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 11:41:20.345974 (+ 548us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48284: BlockingRecv error: recv got EOF from 127.0.0.1:48284 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
I20260806 11:41:20.636459 13506 catalog_manager.cc:1259] Loaded cluster ID: d69cf8610b6543e9a3a58f2eabc91d01
I20260806 11:41:20.636554 13506 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f28ea935f3c842b6a754b86d3806ff38: loading cluster ID for follower catalog manager: success
I20260806 11:41:20.638331 13506 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f28ea935f3c842b6a754b86d3806ff38: acquiring CA information for follower catalog manager: success
I20260806 11:41:20.639606 13506 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f28ea935f3c842b6a754b86d3806ff38: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 11:41:20.645150 13492 catalog_manager.cc:1259] Loaded cluster ID: d69cf8610b6543e9a3a58f2eabc91d01
I20260806 11:41:20.645238 13492 catalog_manager.cc:1552] T 00000000000000000000000000000000 P dd27c41696ba4c2d893b6a4296cb337e: loading cluster ID for follower catalog manager: success
I20260806 11:41:20.647135 13492 catalog_manager.cc:1574] T 00000000000000000000000000000000 P dd27c41696ba4c2d893b6a4296cb337e: acquiring CA information for follower catalog manager: success
I20260806 11:41:20.648430 13492 catalog_manager.cc:1602] T 00000000000000000000000000000000 P dd27c41696ba4c2d893b6a4296cb337e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 11:41:20.865001 13586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:20.860788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48294 (local address 127.21.14.253:42141)
0806 11:41:20.861058 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:20.861063 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:20.861156 (+ 93us) server_negotiation.cc:407] Connection header received
0806 11:41:20.861343 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:20.861346 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:20.861490 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:20.861623 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 11:41:20.861899 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:20.862444 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:20.863412 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:20.863663 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:20.863779 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:41:20.864133 (+ 354us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 11:41:20.864588 (+ 455us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 11:41:20.864731 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 11:41:20.864773 (+ 42us) server_negotiation.cc:299] Negotiation successful
0806 11:41:20.864832 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":75,"threads_started":1}
I20260806 11:41:20.872030 13592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:20.868388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55728 (local address 127.21.14.252:39035)
0806 11:41:20.868642 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:20.868645 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:20.868682 (+ 37us) server_negotiation.cc:407] Connection header received
0806 11:41:20.868767 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:20.868769 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:20.868888 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:20.868980 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 11:41:20.869199 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:20.869779 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:20.870607 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:20.870840 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:20.870978 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:41:20.871256 (+ 278us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 11:41:20.871702 (+ 446us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 11:41:20.871794 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 11:41:20.871852 (+ 58us) server_negotiation.cc:299] Negotiation successful
0806 11:41:20.871903 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":80,"threads_started":1}
I20260806 11:41:20.879696 13586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:20.875369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48308 (local address 127.21.14.253:42141)
0806 11:41:20.875504 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:20.875509 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:20.875959 (+ 450us) server_negotiation.cc:407] Connection header received
0806 11:41:20.876097 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:20.876099 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:20.876178 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:20.876254 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 11:41:20.876674 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:20.877234 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:20.878358 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:20.878714 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:20.878801 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:41:20.879234 (+ 433us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 11:41:20.879332 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 11:41:20.879404 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 11:41:20.879488 (+ 84us) server_negotiation.cc:299] Negotiation successful
0806 11:41:20.879544 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 11:41:20.879894 13592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:20.875594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55734 (local address 127.21.14.252:39035)
0806 11:41:20.875716 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:20.875722 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:20.875940 (+ 218us) server_negotiation.cc:407] Connection header received
0806 11:41:20.876161 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:20.876166 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:20.876223 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:20.876298 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 11:41:20.876697 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:20.877490 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:20.878635 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:20.878984 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:20.879099 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:41:20.879367 (+ 268us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 11:41:20.879440 (+ 73us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 11:41:20.879507 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 11:41:20.879676 (+ 169us) server_negotiation.cc:299] Negotiation successful
0806 11:41:20.879740 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 11:41:20.880379 13599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 11:41:20.875370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32986 (local address 127.21.14.254:42233)
0806 11:41:20.875964 (+ 594us) server_negotiation.cc:206] Beginning negotiation
0806 11:41:20.875968 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 11:41:20.875982 (+ 14us) server_negotiation.cc:407] Connection header received
0806 11:41:20.876101 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 11:41:20.876105 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 11:41:20.876389 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 11:41:20.876538 (+ 149us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 11:41:20.876777 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:20.877567 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:20.878431 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 11:41:20.878725 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 11:41:20.878822 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 11:41:20.879285 (+ 463us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 11:41:20.879675 (+ 390us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 11:41:20.879765 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 11:41:20.880183 (+ 418us) server_negotiation.cc:299] Negotiation successful
0806 11:41:20.880245 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":503,"thread_start_us":77,"threads_started":1}
I20260806 11:41:20.889005 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 13221
I20260806 11:41:20.897622 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 13301
I20260806 11:41:20.905668 21563 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4_zrTV/build/debug/bin/kudu with pid 13379
2026-08-06T11:41:20Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3290 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6515 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (311829 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST